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FOREWORD 


These proceedings contain the papers and posters of the 5 th International Conference on 
Educational Technologies 2017 (ICEduTech 2017), which has been organised by the 
International Association for Development of the Information Society and co-organised by 
the Western Sydney University, held in Sydney, Australia, 11-13 December 2017. 


ICEduTech is the scientific conference addressing the real topics as seen by teachers, 
students, parents and school leaders. Scientists, professionals and institutional leaders are 
invited to be informed by experts, sharpen the understanding what education needs and how 
to achieve it. 


Topics for the ICEduTech Conference: 

• Education in Context: Education in the Network Society, Educational Games, 
Social Media in Education, Home Schooling, Students’ Rights, Parents’ Rights, 
Teachers’ Rights, Student-Safe Searching, School Violence, Education and 
Tolerance for Peace and Education in Developing Countries. 

• Education as Professional Field: Teacher Education, Teachers’ Professional 
Development, Teachers’ Workload, Teacher Support for Grading, Time Tabling, 
Grading, Learning Tools, and Online Learning Software, Teachers’ Learning in 
Communities of Practice, Web-based Communities for Teacher Support, Teachers’ 
Career Planning, Legal and Financial Issues, Conflict Resolution and Mediation, 
Governance and Servant Leadership and Educational Policies. 

• Curricular Evolution: Problem-based Learning, Critical Thinking Skills, Creativity 
Skills, Learning Citizenship, Global Education, Media Literacy / Pedagogy, 
Multicultural Education and Alternative Assessment Methods. 

• Learner Orientation: Student-Oriented Learning, Peer- and Collaborative Learning, 
Learning Strategies: Learn how to Learn, Motivating Students, Recognizing 
Students’ Learning Styles and Special Education. 

• Integrating Educational Technologies: Social Media and Social Networking, The 
Semantic Web 3.0, Podcasting for Broadcasting Video Lectures, Podcasting 
feedback to students, Wiki and blogs in Higher Education, Mobile, Virtual and 
Vicarious Learning and Simulations and Modeling. 

• International Higher Education: Marketing Higher Education as a Business Case, 
Pitfalls and Solutions in Joint and Double Degree Programs, Enculturation and 
International Teacher Accreditation, Web-based, Mobile, Virtual Presence and 
Social Media to Overcome Student Mobility, Blended Learning and Student 
Assessment at a Distance, Student Mobility and Distance Education, 
New-Emerging Standards and Benchmarks for Higher Education, Education, 
Research, Exchange an Capacity Building, 21st Century Academic and Industrial 
Brain Exchange, Academic Salaries, Faculty Contracts, Residence Permits and 
Legal Issues, International Student Exchange Funding Programs: Erasmus Mundus, 
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the U.S. Council on International Educational Student Exchange, and the 
Euro-American “Atlantis” Program, Networks for International Higher Education in 
the Pacific, Australia, Europe, Asian and European Countries and Higher Education, 
Cultural Diversity, Tolerance and Political Conflict. 


The International Conference on Educational Technologies 2017 (ICEduTech 2017) 
received 95 submissions from more than 19 countries. Each submission was reviewed in a 
double-blind review process by an average of four independent reviewers to ensure quality 
and maintain high standards. Out of the papers submitted, 18 got blind referee ratings that 
published them as full papers, which mean that the acceptance rate was 19%. Some other 
submissions were published as short papers and posters. 

Best paper authors from the ICEduTech 2017 conference will be asked to extend their 
papers for inclusion in a special issue of the Journal of Information, Communication and 
Ethics in Society (JICES) (ISSN 1477-996X) and other selected journals and/or books. 

In addition to the presentation of full and short papers and posters, the conference also 
includes two keynote presentations and one workshop entitled “Teacher Dashboards: 
Critical Insights and Informed Design Considerations” by Associate Professor Pedro Isaias 
and Solange Silva, The University of Queensland, Australia. We would like to express our 
gratitude to Associate Professor Pedro Isaias, The University of Queensland, Australia and 
Associate Professor Ana Hoi, Western Sydney University, Australia as our keynote 
speakers. 

A successful conference requires the effort of many individuals. We would like to thank the 
members of the Program Committee for their hard work in reviewing and selecting the 
papers that appear in this book. We are especially grateful to the authors who submitted 
their papers to this conference and to the presenters who provided the substance of the 
meeting. We wish to thank all members of our organizing committee. 

Last but not least, we hope that participants enjoyed Sydney and their time with colleagues 
from all over the world. 


Piet Kommers, University of Twente, The Netherlands 
Ana Hoi, Western Sydney University, Australia 
Conference Co-Chairs 

Tomayess Issa, Curtin University, Perth, Australia 
Pedro Isaias, The University of Queensland, Australia 
Conference Program Co-Chairs 

Sydney, Australia 
December 2017 
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By Associate Professor Pedro Isaias, 
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Abstract 

Evolution is the operative word in higher education, as roles shift, classrooms are 
reinvented and content becomes increasingly more accessible and malleable. At the core of 
these changes is the pervasiveness of educational technology. Higher education is 
progressively being displaced from the traditional classroom and, as it progresses towards 
online settings, it requires the support of technology to facilitate that transference. Within 
the context of higher education, there are numerous technologies that will have a 
revolutionary impact on teaching and learning. In exploring a prospective scenario for 
2020, 2025 and 2030, it is possible to envision a higher education sector that has widely 
adopted innovative Learning Management Systems, adaptive learning technologies, 
Massive Open Online Courses, mobile learning, Artificial Intelligence, activity-based 
technology, Internet of Things and Social Technology. These technologies are expected to 
have profound implications in the traditional learning environments and require thorough 
preparation. Predictions as they may be, the exercise of forecasting provides the present 
with the opportunity to prepare for the future. 


UNIVERSITIES IN 2020S 

By Associate Professor Ana Hoi, 

Western Sydney University, Australia 

Abstract 

2020s are expected to bring new changes and challenges to many disciplines including 
education. It is apparent that the traditional modes of teaching and learning are changing 
and that standard classrooms and particularly university lecture halls are slowly reducing in 
sizes. Worldwide universities are moving into collaborative spaces and are removing large 
classes which once were a norm. Consequently, the content that was delivered once now 
needs to be changed and adopted to suite new requirements. This keynote reviews current 
and emerging modes of content deliveries in STEM (Science, Technology, Engineering and 
Mathematics) fields particularly taking into the account the nature in which content is being 
delivered to students and reviews how over time modes of content delivery and in some 
instances even the content being delivered is being changed. 
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Abstract 

Teacher dashboards can be a powerful tool to provide teachers with critical information 
about students" engagement and performance. The Khan Academy provides a useful 
dashboard that can inform the design of other Teacher's dashboards. Participants will 
experience the Khan Academy environment as well as an example from an application in a 
public School in Brazil. Participants will also have the opportunity to reflect and critique on 
Teacher dashboards requirements and design considerations. 
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BIBLIOMETRIC SCIENCE MAPPING AS A POPULAR 
TREND: CHOSEN EXAMPLES OF VISUALISATION OF 
INTERNATIONAL RESEARCH NETWORK RESULTS 


Eugenia Smyrnova-Trybulska 1 , Nataliia Morze 2 , Olena Kuzminska 3 and Piet Kommers 4 

University of Silesia in Katowice, Faculty of Ethnology and Sciences of Education, Bielska 62, 43-400 Cieszyn, Poland 

~Botys Grinchenko Kyiv University, Ukraine 
3 University of Life and Environmental Sciences of Ukraine 
4 University of Twente, Netherlands 


ABSTRACT 

The authors of the article describe the popular trends and methods as well as ICT tools used for the mapping and 
visualization of scientific domains as a research methodology which is attracting more and more interest from scientific 
information and science studies professionals. The researchers analysed Pajek, one of the programs used for the 
processing and visualization of bibliographic and bibliometric data, within the framework of the implementation of IRNet 
research network project and activities, and presented several examples of visualisation. 


KEYWORDS 

Networking, Bibliographic and Bibliometric Data, Mapping, Visualization. Applications 


1. INTRODUCTION 

Modern science is undergoing enormous changes and transformations. Science 2.0 is a term used to refer to 
processes, trends and phenomena related to the use of new technologies and information and communication 
technologies in science, and in particular, tools, services, publications and online resources. This trend 
stresses the benefits of increased collaboration and cooperation between researchers and scientists. However, 
it refers primarily to the consequences of using these technologies. 

At this stage of Web 2.0 development a popular trend has emerged whereby knowledge resources are 
mapped from public web services. For example, a map of English Wikipedia articles was generated using a 
measurement registering common categories of articles (Holloway, Bozicevic, Borner, 2007). User activity 
specialist services also inspire analysis and mapping of data such as blogs and forums, users’ logs (Bollen, 
Van de Sompel, Hagberg, Bettencourt, Chute. Rodriguez, Balakireva, 2009) etc. The concept of the mapping 
and visualization of scientific domains as a research methodology is presented in Figure 1. 

Scientometric and bibliometric approaches are being increasingly used by some authors to assess the 
evolution and structure of scientific knowledge and R&D output (e.g., Meyer, M. (2004), Wagner, C. S., and 
Leydesdorff, L. (2005), Dietz, J. S., and Bozeman, B. (2005), Adams, J. D. (2006), Hussler, Caroline, and 
Ronde, Patrick (2007)). Normally, studies within this research field (Meyer, M. (2000b), Meyer, M. (2004), 
Wagner, C. S., and Leydesdorff, L. (2005)) aim to appraise the scientific output of individuals, journals and 
even organizations (e.g., effective publication in internationally refereed journals, high citation scores) by 
surveying and analyzing co-authorships and citation indexes. 


2. SCIENCE MAPPING OR BIBLIOMETRIC MAPPING 

The existing methods of information visualization have been successfully adapted in a network environment 
where websites, with some approximation, can be regarded as scientific articles, and hyperlinks as quote 
links. This approach has promoted the rapid development of Webometrics (Zhao, Strotmann, 2008), Osinska, 
2010 ). 
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Science mapping or bibliometric 

mapping 


(a) data sources, 

Science mapping or < b ) units of analysis, 

bibliometric mapping is a ( c ) data Preprocessing, 

spatial representation of how (d) similarity measures that can be used to normalize 
disciplines, fields, specialities, the relations between the units of analysis, 

and individual documents or jgj mapping steps, 

authors are related to one (f) typ es 0 f methods of analysis that can be employed, 
another (g) some visualization techniques, and finally, 

(h) interpretation of results 


Figure 1. Science Mapping Concept 

According to (Wagner, Leydesdorff, 2005), authors within this research field are interested in the increase 
of the interconnectedness of scientists (e.g., Okubo, Miquel, Frigoletto, and Dore, 1992), Luukkonen, 
Tijssen, Persson, Sivertsen, 1993; Zitt, Bassecoulard, Okubo, 2000; Glanzel, 2001; Cantner, Graf, 2006), in 
figuring out patterns of collaboration in general (e.g., Chung, Cox, 1990, Gibbons, Limoges, Nowotny, 1994; 
Katz, Martin, 1997; Dietz, Bozeman, (2005), Hussler, Ronde, 2007, and of international linkages in particular 
(e.g., Stichweh 1996, Schott, 1998) and further analyzing implications of linkages for funding and outcomes 
(e.g. Van Den Berghe, Houben, De Bruin, Moed, Kint, Luwel, Spruyt, 1998; Wagner, Yezril, Hassell. 2000; 
Sequeira, Pacheco, Teixeira: p. 8-16). 

To better understand the structure and dynamics of the development of individual science departments 
and to find a way to identify thematic trends, researchers analyze literature and the paths to cite individual 
publications. Analytical citation has been used in bibliometrics for a long time. The term developed by 
KDViz (Knowledge Domain Visualization) was coined by Ch. Chena in 2001, editor of the leading 
information visualization magazine Information Vizualizatiorf (common abbreviation: InfoViz). Because 
these methods lead to the generation of graphical maps, another name is used in parallel: Mapping Science 
or, less often, scientography. This last one was introduced in 1960 by E. Garfield, founder of the Institute for 
Information Science in Philadelphia (ISI). Over the past 10 years, the domain knowledge visualization 
research has been extended to include information retrieval tasks (Osinska, 2010). 


Different approaches have 
been developed to extract 
networks using the selected 
units of analysis (authors, 
documents, journals, and 
terms). 

Co -author 

Co-word 

Co-citation 


3. SCIENCE MAPPING 

Science mapping or bibliometric mapping is a spatial representation of how disciplines, fields, specialties, 
and individual documents or authors are related to one another (Small, 1999). It is focused on monitoring a 
scientific field and delimiting research areas to determine its cognitive structure and its evolution (Noyons, 
Moed & Van Raan, 1999). There are different important aspects to a science mapping analysis, such as: (a) 
data sources, (b) units of analysis, (c) data pre-processing, (d) similarity measures that can be used to 
normalize the relations between units of analysis, (e) mapping steps, (f) types of methods of analysis that 
can be employed, (g) some visualization techniques, and finally, (h) interpretation of results. 

Different approaches have been developed to extract networks using selected units of analysis (authors, 
documents, journals, and terms). Co-word analysis (Callon, Courtial, Turner & Bauin, 1983) uses the most 
important words or keywords of the documents to study the conceptual structure of a research field. 
Co-author analyzes authors and their affiliations to study the social structure and collaboration networks 
(Ganzel, 2001); Peters & Van Raan, 1991). 

Finally, the cited references are used to analyze the intellectual base used by the research field or to 
analyze documents that cite the same references. In this sense, bibliographic coupling (Kessler, 1963) 
analyzes citing documents, whereas co-citation analysis (Small, 1973) studies cited documents. Other 
approaches such as author bibliographic coupling (Zhao & Strotmann, 2008), author co-citation White & 
Griffith, 1981), journal bibliographic coupling (Gao & Guan, 2009; Small & Koenig, 1977), and journal 
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co-citation (Mccain, 1991) are examples of macro analysis using aggregated data (Cobo, Lopez-Herrera, 
Herrera-Viedma, Herrera, 2011: 1382). 

Interesting research results were received by Spanish authors (Gomez, Ignacio; Vazquez-Cano, Esteban; 
Lopez-Meneses, Eloy, 2016). According to their research, Spain was in 2013 the leading European country in 
MOOC course offerings and is in a leading position worldwide in the number of massive courses offered in 
2014. This prolific activity is being transferred to the educational and scientific world in the form of posts in 
blogs, social networks and web pages, as well as scientific papers and books that attempt to analyze the 
movement from different methodological approaches. To date there is not any research that analyzes the 
bibliometric impact of MOOC movement in the Spanish scientific community. Therefore the objective of this 
research is to perform a bibliometric study of the scientific impact in the form of scientific article or research 
book in journals and Spanish publishers from January 2010 to June 2014. The study was approached from a 
descriptive and quantitative methodology, taking as reference bibliometric indicators of production, number 
of citations, and indicators of visibility according to their impact on different databases: WoS / Social 
Science Citation Index, Scopus, In-Recs, Google Scholar and categorization of Spanish scientific journals 
(ANEP / FECYT). The results show that the impact of the Spanish scientific production in the form of books 
and scientific articles in prestigious international databases (WoS-SSCI / Scopus) is very low, although the 
national impact categorization according to ANEP / FECYT and In-Recs is moderately high (Gomez, 
Ignacio; Vazquez-Cano, Esteban; Lopez-Meneses, Eloy, 2016). 

Mapping and visualization on the TimeLine is comprehensively described and presented in Vieslava 
Osinska (2016). 

The following software products are used for processing of bibliographic and bibliometric data: 

• CiteSpace, CiteSpace II 

• Bibexcel 

• Pajek 

• VOSViewer 

• HistCite 

• XLSTAT 

• Publish or Perish 

• VantagePoint 

• Sci2 Tool 

• UCINET 

Applications used for a typical statistical analysis of large data sets include Statistica, Origin, Matlab, 
Mathematica. In the list above, special note should be made of CiteSpace - a continuously developed 
complete tool supporting all stages of the visualization process from data extraction to validation of the 
resulting graphic configuration. 


4. MAPPING AND VISUALIZATION OF IRNET PROJECT 
NETWORKING ACTIVITIES 

Within the framework of the IRNet project (www.irnet.us.edu.pl ) research is being conducted in several 
WPs which are separate, yet simultaneously connected through interrelated stages, which roughly address our 
several research questions.. A bibliographic database was created containing all the published articles, books 
from 2014 to date by all IRNet researchers (178 publications) according to the following rubrics: names and 
surnames of (co)authors, affiliation, title of publication, type (article, chapter, book, etc.), where published 
(conference proceedings, monograph, journal, etc.), year key words, indexing. 

This database also includes information regarding the number of authors, authors' affiliation, country, 
year, research areas, keywords, the source of publication (e.g. journal, book, etc.) and certain other data. 
Consequently, this dataset enables us to assess the main trends in IRNet scientific publications production. 
The time frame of the analysis is the last 4 years of IRNet existence, in which we have been able to trace its 
knowledge production and dissemination. 

Based on the dynamics of international co-authorships, we will be able to map and trace international 
collaboration patterns and thus infer IRNet geographical influence scope, i.e. its international influence 
(Research Question). 

By means of additional research utilizing information available in the Institute for Scientific Information 
(ISI), namely in the Science Citation Index (SCI), we assess the geographical pattern of the citations of IRNet 
scientific activities, publications, production. This enables us to evaluate to what extent IRNet project 
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scientific production has been cited at the European and world level. 

As the subject matter of the project is related to educational technologies or, to be more precise, to 
e-learning technologies, which is reflected in the participants’ publications, one can assume that an analysis 
of international collaboration and the degree of participants’ involvement in various countries etc., will 
indirectly allow for an evaluation of the objectives of e-learning technology development in each partner 
country and institution. 


5. METHODOLOGY OF RESEARCH 

Among the main indicators of research effectiveness - bibliometric indicators - is a powerful information tool 
to support the development of science. Practical research methods were prepared using the programs: Bibexel 
(metadata analyses), and program Pajek: graph editing and visualization of the graph structure of 
co-authorship - using the method of Kamada-Kawai (Tomihisa Kamada & Satoru Kawai). 

For the study of science as a process of scientific communication, which is relevant to this study, one can 
use the tools of network analysis. As a tool for analysis and visualization of large scale networks we can 
recommend Pajek (Pajek, 1996). However, for complex types of analysis, one must have competencies 
helping one to work with specialized products and to develop programming skills. 

To carry out a network analysis of participants of the project, we will build a network of co-authorship for 
research activity and presentation of results in scientific publications. 

Analyses of data for the purpose of network analysis were performed using Bibexel, starting with 
obtaining initial data for this program. These data (up to the end of 2016) were assembled using built-in tools 
of the scientometric database Web of Science. Data for each participant in the project IRNet are files with the 
extension .txt which were combined using the program Bibexel: in the new window “Type new/file name 
here”, one should enter name which needs to have the merged file, select all the files one wants to merge, and 
run Files -> Append all selected files to another. 

For building a graphics editing program Pajek (http://mrvar.fdv.umlj.si/pajek//) the following objects 
were used: 

1. Networks are the main objects (vertices and lines), and the default extension is: .net. In this case, the 
created file is Analiz.net of the 83 vertices - authors’ articles and links between them, reflecting the 
co-authorship in publications. The file specifies an additional property of vertices - the identity of 
the authors who analyzed the participants of the project (diamond). 

*Vertices 83 

1. 1 "Kommers P." diamond 

2. 2 "Smyrnova-Trybulska E." diamond 

3. 3 "Morze N." diamond 

4. 4 "Noskova T." diamond 

5. 5 "Pavlova T." diamond 

6. 6 "Yakovleva O." diamond 

7. 7 "Sekret I." diamond 

8. 8 "Zavgorodnyi V." diamond 

9. 9 "Yalova K." diamond 

10. 10 "Yashina K." Triangle 

*Edges 
1 2 10 
1 3 8 

2. Vectors - file Analiz.vec, contains 83 entries, where for each vertex (author) its quantitative 

characteristic (real number) - the number of publications is indicated. *Vertices 83 

1 12 2 58 3 53 4 27 5 30 

3. Partitions - file Analiz.clu, where for each vertex the class is defined to which it belongs. In this case, 
there are 10 classes, in line with the countries’ mission project participants: 

1- Australia, 2- Czech Republic, 3 - Netherlands, 4 - Poland, 5- Portugal, 6 - Russia, 7- Slovakia,8- 
Spain, 9 -Turkey, 10 - Ukraine. 

*Vertices 83 

3 4 10 6 6 6 


6 



International Conference Educational Technologies 2017 


To build a network visualization of co-authorship in the program Pajek, it is necessary to download the 
file Analiz.net in the Networks field. Vectors - Analiz.vec Partitions - file Analiz.clu, and select Draw -> 
Network + First Partition+ First vector. 

As for building the visualization of networks linking scientists with special software and settings, the 
project participants’ "labeled" rectangles (diamond) and the corresponding vertices have the signatures - the 
names of the scientists, not the participants - triangles (Triangle); shapes-marks are of different colors 
depending on the country, which represent the authors, and the size that is determined by the number of each 
author’s publications. As can be seen in Figure 2, the scientists have different publication activity, some who 
are not the participants of the project have a greater number of publications than some of the participants. 
The latter can be considered as a basis for widening the circle of participants or an invitation of active 
scientists to other projects. The window where in the text file the relationships of a particular author 
(co-author) is described, can be called using the right mouse button for the selected node. 

As a result of the visualization of the graph structure of co-authorship we have obtained a Kamada-Kawai 
layout (Tomihisa Kamada & Satoru Kawai) (Figure 2). 



Figure 2. Example of Visualization of Co-Authorship with the Countries’ Mission of Project Participants 

Kamada-Kawai Layout 


This made it possible to analyze the contribution (number of publications) of the project participants and 
their collaborators within each country. For example, the largest representation of scientists are from 
Ukraine, representatives from Russia (3 participants) did not attract other sponsors from their country, but 
their activity was high, and there are countries, represented by one party such as Turkey (Sekret I.) and the 
Netherlands (Kommers P.), but the representative from the Netherlands cooperated with other project 
participants actively. 

Flowever, it should be noted that visualization network analyses are often unreadable because they 
contain a large number of nodes with one or two connections. 

The concentration of connections allows for visually accentuating the centers of the analyzed activity, but 
it complicates general perception and makes the label unreadable. To view maps based on the results of the 
network analysis, it is recommended that the concentration of connections be reduced. Therefore, by means 
of the program Pajek a subnet was created consisting only of participants of the project 
(Operations/Network+Partition/Extract SubNetwork). When one starts this, first it is necessary to create a file 
- Part.cls where the elements of the desired cluster should be grouped. In this case it is 42 participants. Such 
a file can be generated automatically, if necessary, changes can be made manually. 

For a network that contains only the data about participants of the project, files and partition vectors are 
formed. You can optionally save this data as separate files or a network in general - with the file extension 
.paj (file/ Pajek project file /save). 

As a result of visualization of the graph structure whose vertices became only the authors of the project, 
we obtain a Kamada-Kawai layout in which the data are not grouped with representation (Figure 3). 
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Figure 3. Example - The Visualizing of the Graph Structure of Co-Authorship of the Project Participants 

Kamada-Kawai Layout 


As can be seen from Figure 3, the distribution of the contribution of the project participants by the 
number of posts is not uniform: the center is represented by more active members. For each network node the 
author's name and the number of publications are added (which also affects the size of the geometric area). 
The color is also allocated to the country offices participating in the project. 

It should be noted that the visualization, basically, is a brief synthesis of the results of the analysis that 
allows us to understand the context of the transition to the data that underpin it. The credibility presented in 
the form of a visualization of the analysis results will be high, if there is a verification of the results of the 
analysis, that is, all the visualization elements retrieved can be traced back to the primary data and methods 
of their processing. 

The results of this visualization can be used to make optimal decisions on the management of information 
resources with the aim of improving processes of scientific communication and the evaluation publications 
contribution of each participant. To determine the scientific impact of the project participants in the 
development of pedagogical science an additional analysis needs to be conducted. 


6. CONCLUSION 

This publication is devoted to the one of the popular trends referred to as mapping of scientific domains and 
visualization of research results in international networks. It can be considered as a research methodology 
which is gaining more and more popularity among scientific information professionals and specialists from 
different disciplines. 

There is an extensive offering of various types of software for processing of bibliometric data - open 
source license as well as commercial license programs. 

The authors of the article, who are researchers of the European IRNet project and international research 
network, described and analysed certain bibliographic results of these activities using methods of mapping 
and visualization of scientific domains. 

The methods for visual representation of scientometric information can serve as a basis for formulating 
preliminary working hypotheses in scientometric data analyses and for presenting the final results of 
analyses, in this case being a statistical analysis (a scholar’s profile), a geospatial analysis (of member states) 
and a network analysis (participation in the project). Scientometric information mapping is closely related to 
the task of preparing forecasts of science development and improving the quality of existing forms of 
collaboration as well as developing new ones. The results obtained give grounds for supposing that the 
development of science branches and disciplines depends, to a large extent, on the existence and 
development of human capital and collaboration forms (we examined collaboration on this project only, but 
collaboration as such is limited to that) and, to a lesser extent, on identifying prospective fields of study and 
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areas of technology. Further research will be pursued, among other things, to design a thematic-based 
collaboration model (constructing scientific e-communication models) and to assess the effect on scientists 
(and individual scientists) and science in general. 
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ABSTRACT 

Due to the competitive and fast-changing nature of external business environments, university students should acquire 
knowledge of how to cooperate, share knowledge, and enhance team effectiveness and individual learning in the future 
workplace. Consequently, the redesign of business courses in higher education merits more discussion. Based on the 
notions of team-based learning (TBL) and flipped classrooms, we proposed a business course model consisting of three 
main phases, which have before-class, in-class, and online-course activities. After implementing these course models in 
two business courses at two public universities in Taiwan, a survey based on social learning and social exchange theories 
was distributed. A total of 262 business undergraduate students participated in this study. The findings show that team 
members’ valuable contributions are important in teams. This has significant impact on knowledge sharing and team 
effectiveness. Knowledge sharing also matters in teams since it is a significant mediator between team members’ valuable 
contributions and team effectiveness. In addition, when team effectiveness is higher, students in this class perceive higher 
levels of individual learning. 


KEYWORDS 

Team-based Learning (TBL), Flipped Classroom, Perceived Team Members’ Valuable Contributions, Knowledge Sharing, 
Team Effectiveness, Individual Learning 


1. INTRODUCTION 

Facing the competition and collaboration of today’s business environment, a primary goal of business 
education is to develop a learning environment that prepares students’ capabilities in their employability 
skills as well as exemplifying appreciation for workplace diversity and respect for ethical values (Mitchell et 
al., 2010; Rutherford et al., 2012). Traditional lecture-based classes have certain pedagogical limitations and 
cannot provide this type of dynamic curriculum and instruction. This has led educators in higher education to 
reform the course design by incorporating technology to flip the teaching and learning environment 
(Al-Zahrani, 2015; Davies et al., 2013). A characteristic of the flipping approach lies in the expectation that 
students study course-related materials before the class (McCallum et al., 2015), whereas class time is 
dedicated to activities designed to promote students’ application of targeted knowledge, abilities and skills 
(Albert & Beatty, 2014; Gilboy et al., 2015). Many of these in-class activities are organized in group-based 
formats in order to foster collaboration, knowledge sharing, and learning processes and outcomes 
(Findlay-Thompson & Mombourquette, 2014). Thus, the main purpose of this research study is to explore 
how the flipped classroom concept is integrated into a business course and how this type of flipped approach 
promotes students’ teamwork and individual learning. 
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2. LITERATURE REVIEW 

The adoption of team-based learning (TBL) provides students with various scenarios and opportunities with 
which to strengthen their readiness for the complicated world that awaits them in the future (Chad, 2012; 
Lightner et al., 2007; Mutch, 1998). Grounded in situations specific to their anticipated workplaces, TBL 
encourages individual students to interact and connect with other team members along with exchanging ideas 
and reaching a consensus (Baldwin et al., 1997; Letassy et al., 2008). TBL cultivates students’ logical 
thinking trajectory and active learning by simulating the types of problems students will encounter in 
workplaces in the future (Kesner et al., 2017). Students move from being passive learners to active learners 
through the process of collaboration and reflection and they take responsibility for a significant amount of 
their own subject area learning and the establishment of targeted competencies (Felder & Brent, 1996; Rasiah, 
2014). 

Transforming the format of traditional transmissive teaching, flipped classroom approaches incorporate 
before-, during- and after-class tasks, that guide students to share their knowledge and to providing reciprocal 
constructive feedback (Wallace et al., 2014). In the flipped classroom, students are required to preview course 
reading or watch videos material before attending a class (Prashar, 2015). During class time, instructors 
prepare more sophisticated work that promotes students’ assimilation of knowledge as well as collaborative 
learning through strategies including role play, discussion, debates and problem solving. The after-class 
activities may consist of various types of assignments, such as practicing individual exercises, reading deeper 
about the course topic, and cooperation on a group project that integrates the in-class teaching and learning 
(Hwang et al., 2015). 

According to social learning theory (Bandura, 1986), students can deepen their learning experience and 
promote active learning through learning from teammates (Gomez et al., 2010). Thus, in the FC-TBL 
environment, in which each individual has more opportunities to observe, feel, and learn about other team 
members’ valuable contributions and performance, students might be more willing to contribute their efforts 
to teamwork as well, and higher team effectiveness can be expected. 

HI: Perceived team members’ valuable contributions are positively associated with team effectiveness. 

Based on social exchange theory, individuals who receive support from the organization or team are more 
likely to provide feedback and contribute to the organization and team in return (Eisenberger et al., 2001). 
Accordingly, this FC-TBL course design may have the potential to foster knowledge sharing in a social 
context. Thus, we posit that when students perceive other team members’ valuable contributions, such as 
sharing knowledge and information with other team members, they are more likely to share what they know 
in teams. 

H2: Perceived team members’ valuable contributions are positively associated with knowledge sharing. 

Researchers have found that knowledge sharing in teams is critical for team effectiveness since team 
members rely on each other (Powell et al., 2004). In addition, through knowledge sharing, team members can 
gain better problem-solving ability (Wellins et al., 1994; Parker, 1990; Nelson & Cooprider, 1996). Hence, as 
previous studies’ findings indicate, we hypothesize that the interaction and communications of knowledge 
and resources among students can benefit team effectiveness (Tsai & Ghosal, 1998; Hansen, 1999; Tsai, 
2000 ). 

H3: Knowledge sharing is positively associated with team effectiveness. 

The perceptions of other team members’ valuable contributions can be influential toward team operation 
and performance (Lindsley et al., 1995; Lester et al., 2002). Some team members’ stronger willingness to 
work may motivate themselves or others to actively participate in team activities (Krackhardt & Stern, 1988), 
and other members would like to do so because all the knowledge and information exchanged become 
important resources or assets among team members (Burt, 2009). When someone is willing to share 
resources and information, thus also making themselves more accessible to other team members, this 
increased accessibility can create a closer and better friendship with each other (Krackhardt & Stern, 1988). 
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Good friendship is helpful for knowledge sharing and transfer (Dhanaraj et al., 2004) and leads to even more 
positive benefits for teams, such as team effectiveness. Consequently, we have the following hypothesis. 

H4: Knowledge sharing will mediate the relationship between perceived team members’ valuable 
contributions and team effectiveness. 

When team members have a stronger team orientation, people are clearer about their roles and jobs in 
teams (Isabella & Waddock, 1994) and learners have higher perceptions of learning from the collaborative 
learning (Gomez et al., 2010). 

H5: Team effectiveness is positively associated with perceived individual learning. 


3. METHODS 

3.1 Sample and Procedures 

Participants were 262 business major undergraduate students at two national universities in Taiwan. All of 
them took the fundamental business courses based on the flipped-classroom designs and team-based learning 
models organized by our research team in two semesters. A total of 240 surveys were returned, and 218 of 
them were valid for further data analysis. The response rate was 83.2%. 

3.2 Measures 

3.2.1 Perceived Team Members’ Valuable Contributions 

This construct was accessed by using the three items which are from a validated survey on asynchronous 
online communications (Wu & Hiltz, 2004). This construct was rated on a 5-point Likert scale. 

3.2.2 Knowledge Sharing 

Knowledge sharing is interaction and knowledge sharing behavior among team members. The construct was 
accessed by using the eight items derived from Nelson and Cooprider’s (1996) and Senge’s (1997) studies. 
This construct was rated on a 5-point Likert scale. 

3.2.3 Team Effectiveness 

The scale of team effectiveness was accessed by the use of 6 items. The scale was originally derived from the 
8 items in two studies: Jeremy and Mahesh’s (2001) and Wang, Yang, and Wu’s (2006). In Wang, Yang, and 
Wu’s (2006) study, team effectiveness was rated on a 5-point Likert scale and had two sub-dimensions: 
performance and quality of work life. One item is about limited budgets in teamwork and is not suitable in 
this research context, so we eliminated the item. 

3.2.4 Perceived Individual Learning 

This construct was accessed by the six items adopted from the prior studies that accessed individual learning 
in an asynchronous computer-supported learning network context (Wu & Hiltz, 2004; Wu et al., 2004; Wu et 
al., 2009). This construct was rated on a 7-point Likert scale. 

According to scholars’ suggestions (Hooper et al., 2008; Hair et al., 2009), standardized factor loadings 
should be at least .50, thus indicating the reliability of the questionnaire scale. After performing the CFA 
testing, we deleted some items of the two constructs: knowledge sharing, and perceived individual learning. 
Overall, we have a total of 20 items in our final questionnaire. 
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4. RESULTS 
4.1 Findings 

Table 1 shows the means, standard deviations, the square root of average variance extracted (AVE) and the 
correlations for all variables in this study. According to Hair et al. (2006), the estimated inter-correlations 
among most of the constructs in this study were less than the square roots of AVE of each construct, and this 
evidence provides support for the discriminant validity of the scales. 


Table 1. Means, Standard Deviations, Validity and Correlations of Variables 


Variables 

Mean 

S.D. 

1 

2 

3 

4 

5 

1. Gender 

.27 

.45 






2. Perceived Team Members’ Valuable 
Contributions 

3.77 

.57 

.05 

(.77) 




3. Knowledge Sharing 

3.92 

.55 

-.01 

47 *** 

(.73) 



4. Team Effectiveness 

3.91 

.56 

.04 

45 *** 

74 *** 

(.73) 


5. Perceived Individual Learning 

4.93 

.89 

.12 

64*** 

57 *** 

.50*** 

(.85) 


Note. The diagonal line of the correlation matrix represents the square root of AVE 
***p < .001, **p < .01, *p < .05; N=218 


Cronbach’s alpha of all variables in this study ranged from .77 to .92. Composite reliability (CR) of all 
variables ranged from .80 to .93. Namely, both Cronbach’s alpha and composite reliability (CR) of each 
construct exceed .70 threshold values, so the internal consistency reliability is acceptable (Bagozzi & Yi, 
1989; Fornell & Larcker, 1981). In addition, the average variance extracted (AVE) of all constructs ranged 
from .54 to .72, exceeding the .50 threshold value (Bagozzi & Yi, 1989; Fornell & Larcker, 1981), so the 
results revealed that the convergent validity for all constructs has been achieved 

We also performed CFA for each of the latent variables and four-factor SEM model measured by 20 
indicators. The result provided the satisfactory model fit indices and evidence of discriminant validity, 
decreasing the potential influence of common methods variance (Podsakoff et al., 2003; Posakoff & Organ, 
1986). The four-factor model represents the measurement model of this study, indicating different 
characteristics and concepts of four constructs in our model (y 2 =513.04. RMSEA=.07, CFI=.92, TLI=.91, 
IFI=.92). 


4.2 Structural Model 

The results of direct effects of perceived team members’ valuable contributions (standardized direct effect 
= .43, /?<.001) on team effectiveness was statistically significant. Hence, hypothesis 1 was supported. To test 
hypotheses 2 and 3, the second conditions of mediation were implemented. The results of the direct effects of 
perceived team members’ valuable contributions on knowledge sharing (standardized direct effect = .47, 
p<.00 I) and the direct effect of knowledge sharing on team effectiveness (standardized direct effect = .76, 
/?<.001, see Figure 3.1) were all statistically significant. Consequently, hypotheses 2 and 3 were supported 
and the second conditions of mediation were completed. 

In addition, the direct effect of team effectiveness on perceived individual learning (standardized direct 
effect = .60, p<.()() I, see Figure 3.1) was significant. As a result, hypothesis 5 was supported. 
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Figure 1. Structural Equation Modeling of the Hypothesized Model 


Note. ***p < .001. **p < .01, *p < .05; N=218 

In order to investigate the indirect effects of dependent variables through the mediator, knowledge sharing, 
we conducted bias-corrected bootstrapping and percentile bootstrapping at a 95% confidence interval with 
5,000 bootstrap samples (Taylor, et al., 2008). Furthermore, we followed the suggestions of Preacher and 
Hayes (2008) and calculated the confidence interval of the lower and upper bonds to see whether zero is 
included in the specific interval to examine if the indirect effect is significant or not. As shown in Table 2, the 
results of the bootstrapping test confirmed the existence of a positive and significant intervening effect for 
knowledge sharing between perceived team members’ valuable contributions and team effectiveness 
(standardized direct effect = .29, p<.001). Because the Z score of the direct effects in product of coefficients 
was .64, which is lowered than the criteria of 1.96 threshold value, the direct effect was proven non-existent. 
Accordingly, hypothesis 4 was supported due to the full mediation found in the aforementioned relationship. 

Table 2. The Mediating Effect of Knowledge Sharing Between Perceived Team Members’ Valuable Contributions and 

Team Effectiveness 


Point estimation 

Product of Coefficients 



Bootstrapping 



Bias-corrected 

95% Cl 

Percentile 

95% Cl 


S.E. 

Z 


Lower 

Upper 

Lower 

Upper 




Indirect Effect 




0.287 

0.073 


3.932 

0.163 

0.458 

0.155 

0.441 




Direct Effect 




0.035 

0.055 


0.636 

-0.069 

0.151 

-0.068 

0.151 




Total Effect 




0.322 

0.087 


3.701 

0.167 

0.51 

0.165 

0.506 


Note. Bootstrapping sample of estimation is 5000 


5. DISCUSSION 

In this TBL flipped classroom, evidence shows that a course design that includes three phases is suitable for 
these business course deliveries and students’ learning in business profession. Further, this FC-TBL course 
design gave students more opportunities to interact and communicate with teammates than did courses in the 
traditional lecture mode, and also more opportunities to create a learning community. Based on our proposed 
hypothesized-model, the construct of “perceived team members’ valuable contributions” greatly impacts 
students’ “knowledge sharing” (H2) and “team effectiveness” (HI). Knowledge sharing is associated with 
team effectiveness (H3). H4 reveals that knowledge sharing plays a mediating role between perceived team 
members’ valuable contributions and team effectiveness. Last but not least, team effectiveness is positively 
related to individual learning (H5). 
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6. CONCLUSION AND FUTURE RESEARCH 

Our research findings are generally consistent with previous studies. It has been highly promoted that 
computer-mediated or blended learning TBL can be viewed as a creative teaching tool to enhance students’ 
learning experience and outcomes (Berge & Collins, 1993; Campbell, 2006; Gomez et al., 2010; Lowry et al., 
2006; Wu & Hiltz, 2004; Wu et al., 2009). Moreover, some researchers stated that the flipped classroom 
models in higher education have become applicable because this teaching approach can take various 
instructional technology and provide learners more opportunities for active learning in class (Abeysekera & 
Dawson, 2015; McLaughlin et al., 2014; Roach, 2014). Thus, we designed the TBL flipped classroom and 
implemented it in business courses for college students and this became our research context. In addition, we 
examined several important variables regarding students’ teamwork and learning in this research context 
based on the notions of social exchange theory (Blau, 1964) and social learning theory (Bandura, 1986). As 
mentioned, the factor of perceived team members’ valuable contributions is responsible for team members’ 
knowledge sharing. Through the process of knowledge transfer and sharing, team effectiveness and 
individual learning can be subsequently achieved, which maintain mutual and reciprocal learning atmosphere. 
The results implied that students in this TBL flipped classroom also interact based on the behavioral concepts 
of social exchange theory (Blau, 1964) and social learning theory (Bandura, 1986). For instance, individuals 
will be more willing to share knowledge when they perceive more valuable contributions from their 
teammates. People not only learn from others; they also help each other to learn in TBL flipped classrooms. 
Along with more knowledge sharing behaviors in teams, the teams can run better, so team can be more 
effective and efficient. In the long run, students thought that they can learn better when they work well in 
teams. That is, whole dynamic processes of conversations and collaboration before, during and after class can 
deepen each individual learner’s understanding of the content subjects through self-directed learning and 
practical application. In brief, these findings are valuable and insightful for instructors to take into 
consideration when they implement TBL flipped classrooms. 

We tried to include some important course factors and variables in this study, but there are still a number 
of limitations. First, the research framework was conducted in only one subject area at two universities. 
Future studies are encouraged to apply and expand this research framework in different course settings. 
Second, quantitative data was used for research analysis in this study, so different research paradigms can be 
adopted for exploring additional information and findings regarding similar instructional designs and topics. 
Third, the research design was conducted by self-report data only. Therefore, we conducted the Harman's 
one-factor test to ensure whether post hoc testing has a serious CMV problem. The results of this test showed 
that no serious threat of CMV bias existed in the study. Future studies are encouraged to come up with 
multiple sources of data collection to tackle this issue. 
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ABSTRACT 

Teaching and learning programming has presented many challenges in institutions of higher learning worldwide. 
Teaching and learning programming require cognitive reasoning, mainly due to the fundamental reality that the 
underlying concepts are complex and abstract. As a result, many institutions of higher learning are faced with low success 
rates in programming courses. This paper presents the design and development of an interactive multimedia simulation 
(IMS) prototype for augmenting the teaching and learning of programming concepts. An initial requirements elicitation 
was conducted with the purpose of obtaining the perceptions of programming lecturers and students regarding the 
programming concepts that present teaching and learning difficulties. The results of this requirements elicitation 
informed the design and development of an IMS prototype. A design-based research methodology was used which 
resulted in both a practical and theoretical contribution, i.e. a fully-functional IMS and an evaluation framework for the 
evaluation of such simulations. 


KEYWORDS 

Programming Concepts, Interactive, Multimedia, Simulation, Design-based Research 


1. INTRODUCTION 

Programming is a major part of the computer science curriculum in many institutions of higher learning 
(Ibrahim et al., 2010; Law el al., 2010). Programming is taught using different programming languages 
(Maloney el al., 2008). In these programming languages, students are taught different concepts that could be 
used to solve specific problems. With the underlying concepts being complex and abstract, programming is 
considered as a hard skill to learn (Shehane & Sherman, 2014; Saeli et al., 2011). The complexities of 
teaching and learning programming concepts lead to lower success rates and high dropout rates in many 
universities worldwide (Shehane & Sherman, 2014; Han & Beheshti, 2010; Tan et al., 2009). Therefore, 
institutions of higher learning can add a competitive advantage by focusing their educational strategies on 
emerging technologies such as interactive and visualization tools in order to match their students’ needs 
(De Gloria et al., 2014; Katai & Toth, 2010). Interactive multimedia simulation is one such tool that can be 
used to augment the teaching and learning. 

Design-based research (DBR) has been defined as a series of approaches that can be used to improve the 
teaching and learning environment with the aim to produce new theories, artefacts and practices (Barab & 
Squire, 2004). This paper presents the design and development of an interactive multimedia simulation for 
augmenting the teaching and learning of programming concepts through three DBR cycles. In general, there 
are many definitions of an interactive multimedia simulation (IMS) depending on the discipline in which it is 
used. For purposes of this study, an interactive multimedia simulation can be defined as a 3D 
(three-dimensional) computer program that combines different multimedia elements, in order to simplify and 
visualize complex and abstract concepts being taught in the classroom, thereby engaging students and 
providing user control through an interactive interface, and provide immediate feedback (Saw & Butler, 
2008; Vaughan, 2006; Alessi & Trollip, 2001). 
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2. RELATED WORKS 

Meerbaum-salant et al. (2013) conducted a study to investigate the learning of introductory programming 
concepts through the Scratch visual environment. Scratch is described as “a visual programming environment 
that is widely used by young people” (Meerbaum-salant et al., 2013: 69). The results showed that the 
learning of programming concepts through the Scratch visual programming environment have improved the 
students' cognitive levels of understanding most concepts. In addition, the number of students’ enrolment in 
programming have increased. Esteves et al. (2009) conducted a study to examine the use of Second Life (SL) 
for problem-based learning in programming at the University of Tras-os-Montes e Alto Douro (UTAD), 
Portugal. SL is described as a 3D online virtual world (Esteves et al., 2009). The findings showed that SL can 
benefit the teaching and learning of a programming language for novice students. 

Tan et al. (2009) conducted a study to investigate the learning difficulties in programming concepts for 
undergraduate students. Accordingly, undergraduate students were invited to participate in a web-based 
questionnaire. An analysis was conducted and the results had shown that many undergraduate students find it 
difficult to design a computer program that can solve a specific problem. Yuen (2006) conducted a case study 
to investigate how an interactive simulation can be used to improve the teaching and learning of 
programming. The findings from the case study revealed that students were encouraged to think and 
construct their own solutions through the use of an interactive simulation. 

Interactive multimedia simulations were successfully utilized in domains such as medicine and 
engineering. However, an interactive multimedia simulation (IMS) in this study is distinct from other 
simulation methods as it was explicitly designed and developed to augment the teaching and learning of 
programming concepts in institutions of higher learning. Moreover, this IMS is presented through a computer 
and can be used without any internet connection. Also, the IMS is portable to enable students to copy and use 
it anywhere, anytime. 


3. COGNITIVE THEORY OF MULTIMEDIA LEARNING (CTML) 

Mayer and Moreno (2003) drew attention to how people learn in a multimedia environment as learning relies 
greatly on humans’ cognitive system. This relates to how much information people can take and process into 
the brain without overloading their cognitive system. The works of Mayer and Moreno (2003) discuss three 
principles of cognitive theory in multimedia learning (CTML), namely: dual channel, limited capacity and 
active processing. Dual channel has been defined as an information processing system which consists of two 
channels to process auditory and verbal information separately. In a limited capacity, the capacity of 
processing auditory or verbal information in human’s memory is limited, though, the authors stated that 
presenting pictures and words in a human’s memory is unlimited. Active processing refers to the active 
processing principle that shows that learning “requires substantial cognitive processing in the verbal and 
visual channels” (Mayer & Monero, 2003:44). Ligure 1 presents the CTML diagram that illustrate the five 
cognitive processes involved in multimedia learning. 



Figure 1. Diagram Presentation of a Cognitive Theory of Multimedia Learning (CTML) Adopted from Mayer (2010:545) 
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The cognitive processes shown in Figure 1 include the selection of words from a multimedia presentation 
in working memory, selection of pictures from a multimedia presentation in working memory, organizing the 
selected words into a verbal representation in working memory, organizing the selected pictures into a 
pictorial representation in working memory, and integrating the verbal and pictorial representations with 
previous knowledge activated from long-term memory (Mayer, 2010). Several studies made emphasis that 
learning involves cognition (Sweller el al., 2011; Homer el al ., 2008; Chipperfield. 2006). Hence, Mayer’s 
CTML guided the design and development of the IMS prototype in this study. 


4. RESEARCH METHODOLOGY 


The underlying paradigm of this study is design-based research (DBR), due to the evidence in the literature 
that demonstrates its potential as a methodology suitable to both research and design of technology-enhanced 
learning environments (Van Wyk & De Villiers, 2014; Amiel & Reeves, 2008; Wang & Hannafm, 2005). 
Some of the features of DBR as stated in related works are: 

Problem-solving: DBR is a problem-solving methodology as it focuses on issues related to 
practical real-world problems (Van Wyk & De Villiers, 2014; Barab & Squire, 2004). 

Appropriate for complex real-world environments: DBR takes place in real-life settings where 
teaching and learning happen. Therefore, the results obtained from the research provide a sense of 
validity and can be used to inform and improve educational practice (Barab & Squire, 2004). 

Integrative: DBR is integrative as it allows the use of mixed methods in a research to obtain quality 
and effective results (Van Wyk & De Villiers, 2014). 

Iterative: DBR involves multiple iterations. Thus, the process of testing and refinement of 
prototypes can be repeated in number of cycles until a specific version is accepted (Amiel & 
Reeves, 2008). 

Collaborative and participative: In DBR. the researcher(s) collaborates with participants 
throughout the analysis and design process to obtain an effective outcome (Amiel & Reeves, 2008; 
Barab & Squire, 2004). 

Dual outcomes: The outcomes of DBR are practical and theoretical. The practical outcomes are in 
the form of innovative products or interventions while the theoretical outcomes are a sets of design 
principles or guidelines (Van Wyk & De Villiers, 2014; Barab & Squire, 2004). 

This study implemented a mixed methods approach (qualitative and quantitative). These methods include 
surveys, prototyping and heuristic evaluation. The initial problem that led to this research is low success rates 
and high drop-out rates in programming courses faced by institutions of higher learning. This study extracted 
the 2013 to 2016 success rates report for high impact programming subjects at TUT (Tshwane University of 
Technology), Faculty of Information and Communication Technology (ICT), Computer Science department. 
The lowest success rate percentage obtained in this report was 14% in Technical Programming IB during 
2015 (Tshwane University of Technology, 2017). The low success rates in programming subjects are by no 
means unique to TUT, but also exist in other institutions of higher learning worldwide (Sarpong el al., 2013; 
Tan et al., 2009). 

Through the application of DBR in this study, it was significant to initially identify programming 
problems from a real-world educational setting. Hence, this study conducted requirements elicitation to 
collect preliminary data. The data for requirements elicitation was collected through survey methods 
(semi-structured interviews and questionnaires). The researcher conducted semi-structured interviews with 
twelve programming lecturers. Also, a questionnaire was used to collect data from sixty programming 
students. The main aim of involving both programming lecturers and students in requirements elicitation was 
to obtain their perceptions on the difficulties of teaching and learning programming concepts. A further aim 
was to obtain their views on the proposed solution (IMS) and how it should be designed to alleviate these 
issues of teaching and learning programming concepts. The findings of the requirements elicitation have 
shown that both programming lecturers and students perceived three main programming concepts to have 
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high levels of difficulties. These programming concepts were data types, control structures and array data 
structures. The outcome of the requirements elicitation directed the study to design and develop the proposed 
IMS prototype. 


5. APPLYING DESIGN-BASED RESEARCH FOR DESIGNING AND 
DEVELOPING THE IMS 

Following the outcome of requirements elicitation, a DBR model was adopted and applied in the design and 
development of the IMS prototype. DBR has a cyclic approach. The design and development of an IMS 
prototype was implemented through three DBR cycles, of which each cycle comprised of the five steps of the 
DBR model. The five iterative steps in each DBR cycle are: problem analysis of a complex problem within 
real-world, design a solution, development a solution, evaluation in practice and reflection (Van Wyk & De 
Villiers, 2014). Figure 2 illustrates the DBR model adopted in this study to design, develop and evaluate an 
IMS prototype. 


PROCESS OUTCOMES 



Figure 2. A Generic Model for Design-based Research (DBR), Adopted from Van Wyk and De Villiers (2014) 

The explanation of each step of the DBR model is as follows: 

1 . Problem analysis - a practical problem is identified within a context and relevant literature is 
reviewed to determine the importance of the problem. Problem analysis is conducted to guide 
the design of a solution. 

2. Design a solution - an initial design is proposed to address the problem identified in Step 1. 

3. Develop a solution - A prototype is developed guided by the existing literature. 

4. Evaluate - a prototype is evaluated to determine its effectiveness. 

5. Reflection - a researcher reflects on the outcomes of Stepl to 4 which leads to the enhancement 
of the proposed solution. 
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To elaborate on how the DBR model was implemented in each DBR cycle for the design and 
development of an IMS prototype, the following subsections detail the steps of the three DBR cycles. 

DBR Cycle 1: 

1. Problem analysis 

The IMS aimed at addressing the difficulties associated with the teaching and learning of 
programming concepts. According to the results of requirements elicitation, the abstraction and 
complexity of programming concepts contribute to the difficulties of teaching and learning. 
Furthermore, programming lecturers and students suggested that the IMS design should include 
features such as visualization, real-world examples, interaction and immediate feedback. 

2. Design a solution 

The findings of requirements elicitation identified three basic programming concepts with high 
levels of difficulties as indicated by both programming lecturers and students. These three basic 
programming concepts are data types, control structures and array data structures. As a result, the 
storyboard scenarios were designed to address the abstraction and complexity associated with 
teaching and learning of these three basic programming concepts. The outcome of this step informed 
the development of the solution. 

3. Develop a solution 

Guided by the literature, a low-fidelity prototype in the form of a storyboard was drawn on paper to 
demonstrate the scenarios designed in Step 2. Generally, a storyboard illustrates all the items that 
will be heard, seen or experienced by the user in the system or tool. Accordingly, the storyboard 
drawings used real-world examples to illustrate the three basic programming concepts (data types, 
control structures and array data structures). The storyboard comprised of scenes such as the menu 
interface, introduction video, practical exercises and assessment scene. Figure 3 presents an example 
of the storyboard scenes. 



Figure 3. Storyboard Scenes Indicating the Main Menu and Sub Menu of the IMS 


4. Evaluate 

The storyboard was evaluated by the two research supervisors for its effectiveness. Both supervisors 
have expertise in computer science education, multimedia design and simulations. The outcome of 
the evaluation led to the refinements of the storyboard scenarios formulated in Step 2. 
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5. Reflection 

The researcher reflected on the outcomes of Step 1 to 4 which provided the need to do further 
improvements on the logical progression of scenes in the storyboard. Also, the storyboard scenarios 
were still inappropriate. The outcome of this step led to a second cycle of problems analysis, 
designing a solution, developing a solution, evaluation and reflection steps. 

DBR Cycle 2: 

1. Problem analysis 

Certain issues were identified from DBR Cycle 1 which led to this cycle. The issues identified were 
inappropriate storyboard scenarios and the storyboard scenes were not arranged in sequence. The 
outcome of problem analysis in this step prompted the design of a solution. 

2. Design a solution 

Guided by the existing literature, the content in the storyboard scenarios was refined and the 
storyboard scenes were arranged in sequence. The evaluation framework for evaluating the IMS 
prototype was also designed. The evaluation framework comprised of three categories, namely: 
Instructional design. General usability and Interactive multimedia simulation design. Each category 
had a number of criteria and evaluation statements. This evaluation framework is an additional 
theoretical output of the overall study, but falls outside the scope of this paper and hence it is not 
described in detail. 

3. Develop a solution 

Deriving from the improved storyboard in the previous step, a high-fidelity prototype was developed 
in the form of a fully-functional interactive multimedia simulation (IMS). In the same way as the 
storyboard, the IMS prototype comprised of three basic programming concepts (data types, control 
structures and arrays data structures). The following software and hardware were used in the design 
and development an IMS prototype: 

Software: 

■ The Autodesk 3Ds Max 15 application was used for modelling 3D graphics and creating 
animations. 

■ Unity 3D 4.6.2 Pro version game engine was used for designing and developing the IMS 
prototype scenes, integrating multimedia elements, scripting and for publishing the IMS 
prototype. 

■ Adobe Photoshop CS6 was used for designing the 2D graphics. 

■ Adobe after effects CS6 was used for designing the introduction videos of each 
programming concept in the IMS prototype. 

■ Adobe Audition CS6 was used for audio editing. 

Hardware: 

■ Desktop computer/laptop was a platform used for designing, developing and publishing the 
IMS prototype. 

■ A keyboard was used for input and interaction with the IMS prototype environment 
throughout the design and development process. 

■ A mouse was used for input and interaction with the IMS prototype environment 
throughout the design and development process. 

■ The audio recording equipment was used to record voice narrations. 

■ Multimedia speakers were used for audio output. 
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Figure 4 is a screen shot of the IMS prototype scene, it shows a man called Bob sleeping in the bedroom 
while the clock alarm is ringing to wake him up. This screen shot illustrates how real-life events relate to an 
IF-statement in control structures programming concept. If the statement shown in the flowchart is true, the 
user should press “enter button” from the keyboard to wake Bob up. 



IF....STATEMENT 


An IF STATEMENT executes a statement if the condition is true 
And if the condition is false, nothing happens. _ 


Wake Bob up! 


End 


Figure 4. IMS Prototype Scene Showing a Man Sleeping and the Alarm Is Ringing to Wake Him Up 


4. Evaluate 

Heuristic evaluation was conducted to evaluate the IMS prototype. Heuristic evaluation is an 
inspection method where-by a small number of experts apply a set of principles to judge aspects of 
an interface or a specific tool in order to improve its appropriateness (Madan & Dubey, 2012). The 
evaluation framework designed in Step 2 was used for heuristic evaluation to evaluate the IMS 
prototype. Heuristic evaluation was performed by nine expert evaluators. The expert evaluators were 
three instructional designers, three usability specialists and three interactive multimedia simulation 
designers. Each group of three experts evaluated a specific category on the evaluation framework. 

5. Reflection 

During heuristic evaluation, the nine expert evaluators identified several design and contextual 
issues in the IMS prototype. Some of these issues were lack of hints and narrations to guide the user 
while performing some tasks on the IMS prototype, inappropriate programming content, poor 
navigation and inappropriate animations. Additionally, a new set of criteria was suggested on the 
evaluation framework itself. The outcome of this step led to a third cycle of problems analysis, 
designing a solution, developing a solution, evaluation and reflection. 


DBR Cycle 3: 

1. Problem analysis 

As mentioned on the reflection step of DBR Cycle 2, several design and contextual problems were 
identified by expert evaluators. In addition, a new set of criteria were proposed on the evaluation 
framework. Accordingly, literature was reviewed to guide the design of the new set of criteria for 
the evaluation framework. 

2. Design a solution 

The new set of criteria such as relevant subject matter, fostering of germane load and artistic 
suitability were designed and incorporated into the evaluation framework. The new revised 
evaluation framework comprised of the same three categories as mentioned in DBR Cycle 2 but. 
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with a new set of additional criteria and associated evaluation statements as recommended by the 
nine expert evaluators. 

3. Develop a solution 

The contextual and design issues such as lack of hints and narrations to guide the user while 
performing tasks, inappropriate programming content, poor navigation and inappropriate animations 
on the IMS prototype as previously identified by expert evaluators in DBR Cycle 2 were addressed 
and improved upon. The outcome of this step prompted a second round of heuristic evaluation to 
ensure the effectiveness of the enhanced IMS prototype and evaluation framework. 

4. Evaluate 

The second round of heuristic evaluation was conducted using the improved evaluation framework 
to evaluate the enhanced IMS prototype. The heuristic evaluation involved the same nine expert 
evaluators from DBR Cycle 2. 

5. Reflection 

The findings of the second round of heuristic evaluation still indicated some minor design issues on 
the IMS prototype such as spelling errors, inappropriate text and animations, and low quality 
graphics. As a result, the researcher refined the IMS prototype to address these design issues. The 
outcome of this cycle lead to the improved IMS prototype and evaluation framework. By now, the 
prototype has been refined into a fully-functional IMS. 


6. DISCUSSION 


It was an aim of this study not just to find a solution to the main problem but also, to document the 
procedures that were followed to obtain the solution. The three DBR cycles presented in the previous section 
described the application of the DBR model to design and develop an IMS porotype for augmenting the 
teaching and learning programming concepts. To indicate how this research adheres to the features of DBR 
presented in Section 4, these features are revisited with an explanation of how each was applied (see Table 
1 ). 


Table 1. The Application of DBR Features in This Research 


DBR Features 

Application to this research 

Problem-solving 

Teaching and learning programming require cognitive reasoning, mainly 
due to the programming concepts being complex and abstract in nature. 
This research had focused on solving a real-world practical problem by 
designing and developing an interactive multimedia simulation (IMS) to 
augment the teaching and learning of programming concepts. 

Appropriate for complex 
real-world environments 

The institutions of higher learning are complex environments that are 
usually faced with similar teaching and learning problems. Therefore, the 
results obtained from the use of an IMS to augment the teaching and 
learning of programming concepts in this research can be significant to 
inform similar studies. 

Integrative 

This research implemented mixed methodologies such as survey, 
prototyping and heuristic evaluation in order to obtain effective results. 

Iterative 

As indicated in the previous section, three cycles of iterative process of 
analysis, design, development, evaluation and reflection was followed by 
the researcher to design and develop an IMS. 
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Collaborative and 
participative 

Initially, the researcher collaborated with programming lecturers and 
students with the purpose of obtaining their perceptions regarding the 
programming concepts that present teaching and learning difficulties. 
Additionally, throughout the IMS design, development and evaluation 
process, the researcher collaborated with programming lecturers, 
instructional design experts, usability experts and interactive multimedia 
design experts. 

Dual outcomes 

The final output of this study is a fully-functional IMS and an evaluation 
framework for the evaluation of such simulations. 


The evaluation framework was mentioned in this paper due to its function in the study and the fact that 
the refined framework is the theoretical contribution of the study, but details of the evaluation framework 
falls outside the scope of this paper. 


7. CONCLUSION 

This paper presented the design and development of an interactive multimedia simulation (IMS) for 
augmenting the teaching and learning of programming concepts. A DBR model was adopted from literature 
and implemented through three DBR cycles to design and develop an IMS prototype. The results of this 
study are a practical contribution in the form of a fully-functional IMS and a theoretical contribution in the 
form of an evaluation framework. Therefore, this study recommends the use of DBR in educational research 
in order to improve both theory and practice. 
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ABSTRACT 

The increasing popularity of tablets in society generally has sparked much interest in their educational potential and while 
a number of studies on the use of tablets in schools have been conducted world-wide most of these have been conducted 
in English-speaking and well-resourced education jurisdictions such as the UK, Australia, New Zealand and the USA. 
Studies conducted on the use of tablets in non-English speaking and educationally under-resourced countries are less 
wide-spread. This research conducted as part of an EU Erasmus-l- Project (Micool), focusses on the introduction of iPads 
in a remote, rural region of Portugal where persistent under investment in education, particularly educational IT, has been 
acutely felt for almost a decade. Using a case study methodology this study reveals how many of the benefits associated 
with using mobile technologies in other educational jurisdictions were also replicated here, and how innovative teachers, 
despite working within a very traditional and rigid system, used the devices to suit their specific curriculum and 
classroom needs. Furthermore, this study will also reveal how international events such as the financial and economic 
collapse of 2008/2009 negatively impacted national education policies in a European country, the effects of which were 
acutely manifested in this school, particularly when it came to IT provision and support. 


KEYWORDS 

iPads, Mobile, Erasmus-l-, Intercultural, Tablets, ICT 


1. INTRODUCTION 

An increasing number of European countries have identified mobile computing devices, particularly Tablets 
as a focal point of their national strategies for education (Horizon Report, 2014). These multifunctional 
“always connected devices” allow simple tools and applications to be easily integrated into classroom 
activities with no need for involvement of IT support (Horizon Report, 2011). The increasing popularity of 
tablets in society generally has sparked much interest in their educational potential and research work has 
illustrated numerous benefits including their potential to enhance learning (Burden et al., 2012); their 
contribution to the development of teamwork and self-directed learning (Ciampa 2014); their motivational 
attributes (Clarke and Svanes 2012); their use as an assessment tool (Clarke and Luckin, 2013); and their role 
in advancing C21st teaching and learning (Melhuish and Falloon, 2010). While such studies documenting 
tablet use have been conducted world-wide most of these have been conducted in English-speaking and 
well-resourced education jurisdictions such as the UK, Australia, New Zealand and the USA. Studies 
conducted on the use of tablets in non-English speaking and educationally under-resourced countries are less 
wide-spread. This research conducted as part of an EU Erasmus-l- Project, known as Micool 
(www.micool.ie), attempts to redress this imbalance by focusing on the introduction of tablets (iPads) in a 
school located in a remote, rural region of Portugal where persistent under investment in education, 
particularly educational IT, has been acutely felt for almost a decade. Research evidence emerging from this 
study adds to the growing body of evidence that tablets when used effectively can enhance teaching and 
learning in different cultural contexts which will be of interest to policy-makers who may question the 
universality of their appeal. Furthermore, by using a case study methodology and “thick description” 
(Denzin, 1989), to document the research findings, this study will illustrate how in the hands of conscientious 
and innovative teachers, tablets are adaptive devices which can be shaped to fit local and national educational 
contexts. Finally this study will also reveal how international events such as the financial and economic 
collapse of 2008/2009 negatively impacted national education policies in a European country, the effects of 
which were acutely manifested in this school, particularly when it came to IT provision and support 
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2. BACKGROUND AND CONTEXT 

“Santo Redentor” (Portugese for “Holy Redeemer”) is a public school located in a remote rural area in 
central Portugal. The school attracts students from villages and rural communities located within a 30 km 
radius. There is not much industry in the area and therefore the majority of students come from families that 
are not well off, but who nonetheless place a high value on education because of the possibilities it affords 
their children to make a better life for themselves. The region was particularly affected by the era of austerity 
as a result of the worldwide financial and economic collapse in 2008/2009, with many families forced to 
emigrate either abroad or to the main urban centres to seek out a living. The effects of this can be clearly seen 
in Santo Redentor’s school numbers which over the last eight years have dropped by almost 35% from over 
1,000 students to approximately 650 students today. 

Santo Redentor is a partner in the Micool (Mobile Intercultural Cooperative Learning) Erasmus+ Tablet 
Project. Involving 6 European countries this two year project (2015-2017) is designed to promote the uptake 
and use of mobile technology in schools. In the latter part of the first year of the project Santo Redentor 
gathered sufficient funds to purchase 22 second-hand iPads. While the school caters for students from 
kindergarten to upper secondary (ages 3-18), with a unit also dedicated to students with special needs, the 
devices were used mainly, although not exclusively, in the secondary school. In March 2015, effectively 15 
months after the school had acquired its iPads Santo Redentor’s secondary school teachers and students 
participated in a research case study designed to document its experiences with using tablets and the benefits 
and challenges it encountered along the way. 

A case study methodology was chosen because of the opportunity it provides to explore a single entity or 
phenomenon known as “the case” (Yin, 1994; Merriam, 1988) bounded by time and activity (a program 
event, process, institution or social group) through in-depth detailed data collection methods utilising “thick 
description” (Denzin, 1989). By its very nature thick description illuminates the context under study and 
allows the reader to enter into the environment and life of a culture as portrayed in the thoughts of the people 
who live there. The main data gathering tool in this study was a series of individual qualitative interviews 
conducted with a total of seven teachers including the school principal (n=7). In addition six focus group 
interviews with students comprising three participants per group (n=18), all aged 16-18 years, were also 
completed. Both teacher and student interviews were conducted using a structured interview format divided 
into key sections encompassing “Pedagogical Benefits”, “Lesson Planning”, “Interactivity and Group Work”, 
“Participants’ Perspectives on Tablet Devices compared to other School Technologies” and “How and in 
what way Tablets were affecting Change?” To protect the anonymity of the participants a pseudo name for 
the school (Santo Redentor) will be used in this paper and teachers will be referred to as “Teacher A, B, C,” 
etc. Student Focus Groups have been labelled “Focus Groups 1 to 6”. In accordance with rigorous qualitative 
research protocols, all interviews were audio recorded, fully transcribed and thematically analysed using the 
structured categories listed above to group the study’s key findings together. In order to ensure rigour and 
validity to this process the thematic analysis was guided by Matthew, Huberman and Miles (1994) structured 
approach to qualitative analysis who defined this process as consisting of three concurrent flows of activity - 
namely, data reduction, data displays and conclusion drawing/verification. Based on following this protocol 
a number of key thematic findings emerged, the most important of which will now detailed in the following 
sections. 


3. KEY FINDING I: ASSESSMENT, DIFFERENTIATION AND SECOND 
LANGUAGE LEARNING 

Research interviews highlighted the predominant role that assessment plays in the Portuguese education 
system. The importance of testing and preparation for exams featured strongly with both teachers and 
students indicating that students were being constantly tested on a weekly basis in practically every subject to 
test out their content knowledge and preparedness for state examinations. Therefore much of the dialogue and 
discussion about the benefits of tablets in learning was framed in terms of their usefulness for testing and 
examination preparation purposes. Given this scenario it is hardly surprising that apps linked to assessment 
and feedback such as ‘Kahoot’ (Kahoot.com), ‘Socrative’ (Socrative.com) and ‘Padlet’(Padlet.com) were 
identified as the apps most widely in all classes with iPads. These apps were liked by both teachers and 
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students as they helped to ease the intensity of lessons, and made for livelier and more energetic classrooms 
by breaking up the monotony of the “teach-content-test” approach. They also introduced an element of fun 
into classrooms which made learning much more dynamic. This in turn led to greater motivation and interest 
in learning as these observations indicate: 

“The benefits are more motivation and interest. When using iPads they are interested in what they are 
doing. And because they are having fun at the same time school is not as boring as they all think...you know 
being in class and having to listen to the teacher again and again. So they are more motivated and I think 
that makes it more effective. ” 

Teacher C 

“They create a different dynamic in the class.... We can chat and we can talk with other when using the 
iPads so we can joke and have fun while learning in class which makes us more relaxed and more concerned 
about what we are going... We learn faster that way while having fun. ” 

Focus Group 6 

The dynamism and fun elements that the iPads introduced into learning appeared to be related to the 
opportunities they opened up for group work. This was particularly the case when it came to using the tablets 
for researching and completing projects. With iPads available to them in classrooms teachers were more 
inclined to encourage students to work in groups to either conduct research or work on projects. This was 
seen as expanding their horizons and helping them to move beyond a reliance on textbook and teacher 
knowledge, thereby encouraging them to become more independent learners: 

“I think when you have the iPads that instead of the teacher having to explain everything you can say 
“look you ’re going to do research on this now” and I find this fosters groups work and opens up possibilities 
for them to work together rather than individually. And it’s good because it moves them beyond the textbook 
as textbooks are very superficial compared to the research they can do with Google. ” 

Teacher F 

“I think it brings a new dynamic to the classroom. It totally does. You ’re not only focussed on the board 
and what the teacher is saying, You have a whole new experience and knowing you ’re working with the 
iPads, you’re expanding your horizons, searching for new things and enlarging your knowledge. I believe it’s 
very good. ” 

Focus Group 2 

As students became more accustomed to using tablets to support classroom learning activities teachers 
also noted how they became less reliant on them and were becoming more independent learners. This was 
viewed very positively especially for those students aspiring to go to university. Another added benefit of this 
development is that it allowed teachers to devote more time in class to less academically able students, hence 
facilitating differentiation. One teacher noted how the iPad suited kinaesthetic learners in particular while 
another teacher commented on how the iPads had helped her to use “more diversified methods so that the 
work the students do is now more independent and not so teacher-based”. She went on to explain how the 
iPad facilitated both the academically advanced student and the academically challenged students 
simultaneously. Using second language learning as an example, she explained how she could now support 
struggling students to complete tasks, while setting additional tasks for more able students: 

“Imagine we ’re doing an exercise on Padlet like I was using in class today where students were required 
to write and upload pictures about a daily routine. Some students struggle with this and needed my help 
while the brighter students finished it quickly. So I could just say to them ‘Okay now you can practice your 
English using Duolingo ” and as they all have an account they just switch over to it. So that kept them busy 
while the others were still concluding. That without the iPads would have been impossible. These good 
students would have stayed there looking at the ceiling or talking to each other and disturbing the class. ” 

Teacher A 
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Given the importance of second language learning, particularly English in the Portugese Education 
system, it is hardly surprising that both teachers and students reported that iPads were used quite extensively 
in this field. “Phraesal Verb Machine” (https://itunes.apple.com/us/app/phrasal-verbs- 
machine/id593374912?mt=8) which was used for learning grammar rules and syntax was one of the most 
popular apps in this domain with some students also reporting the occasional use of "Duolingo” 
(https://www.duolingo.com/). Unusually students did not report ever using the iPads in-built recording 
facilities to practice and listen back to their pronunciation. Nor did they ever try to use this facility on their 
own smartphones while at home to help reinforce their spoken English or German language learning. 

However one teacher observed that as she became more experienced with tablets she began to move away 
from using these specific language learning apps as she felt they were overly drill and practice focussed with 
too much emphasis on individualistic learning. To counteract this she moved towards using more 
collaborative learning apps like ‘Padlet’ where students could work in groups and cooperate while learning 
vocabulary and completing assessments. This was a welcome development as her students observed: 

“I really like the Padlet because we can search, we can write down our ideas and then share it with the 
rest of the class instead of just doing those games like phrasal verb which is a really simple game. So I think 
the Padlet is the one that we’ve been using a lot recently, and that’s the one we like the most. ” 

Focus Group 4 


4. KEY FINDING 2: THE IPAD AS A PREFERRED PEDAGOGIC 
TECHNOLOGY TOOL 

The research interviews revealed that compared to previous school based technologies, most teachers 
believed the iPad was a more intuitive tool and therefore easier to use. Much of this can be attributed to the 
affordances of the device itself as features such as size, mobility, quick start-up time, the availability of 
learning apps and instant internet access contributed to its appeal as a classroom teaching and learning tool. 
Teachers also reported that they had hardly used the schools dedicated computer rooms at all since the iPads 
arrived, although it should be noted that they also reported that they had used the computer room infrequently 
anyway prior to acquiring tablets. This was due to the fact as one teacher said that the “computers are old, 
very slow and some of them don’t even work’’. But even if this was not the case most teachers indicated a 
preference for investing in more iPads for the school rather than computers. As one teacher observed: 

“This year I haven’t gone to the computer room yet and we are now in March. I prefer iPads because I 
can use them in my room without having to go to a specific computer room. My lessons now are more 
interactive because in the classroom 1 can talk and see what they are doing while in the computer room I 
tend to sit at my desk looking at what they are doing but not interacting with them. With the tablets it’s 
different because you are walking and talking and interacting more. ” 

Teacher C 

Although existing computers were deemed to be old and slow there was a general consensus that as a 
school they were quite lucky as a member of the administrative staff had an interest in technology and acted 
as an ad-hoc, on-site expert in maintaining the computers and school network. Alongside the IT teacher he 
helped to keep the school computers, which consisted of a computer (or laptop) and data projector in every 
class, the two main computer labs (rooms) and school library computers, operational. However if the school 
had not had this expertise on site their school computers would have ceased to function a long time ago. Due 
to the cutbacks experienced during the austerity era the school had been unable to update any of its 
computers which were now more than 10 years old and the only infrastructural investment it had made in that 
period was in its Wi-Fi system which was quite good. However many teachers expressed disappointment 
about the lack of support at government level for school technology when it came to upgrading machines or 
providing funding for a dedicated technical support person in schools. This lack of support from the Ministry 
for Education was best encapsulated by one teacher who queried why, if the Ministry could support all other 
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areas of school life such as canteen staff, security staff, cleaners and office staff, why not School IT? "How is 
it possible ”, this teacher said, "to keep an IT system working with nobody employed to keep it going? I think 
it’s not possible. ” 

It is probably due to the investment that Santo Redentor had made in its Wi-Fi and broadband 
infrastructure that teachers reported experiencing few, if any, technical difficulties with the iPads. The only 
issue raised related to battery issues if the devices had not been put back on charge once a class had finished 
using them. The bulk of technical and administrative tasks associated with iPads such as purchasing apps, 
updating and storage management were performed by the two teachers who represented the school on the 
Micool Project. They also ran the booking and timetabling system whereby teachers could book the iPads 
online via the school Moodle VLE platform each week for class use and this system was reported to be 
working well. 

When the school initially became involved in the Micool Project the Principal and the project’s teacher 
representatives hosted an information session for staff to demonstrate the iPads capabilities. All teachers 
were encouraged to take the tablets home with them for a number of weeks over school holiday periods to 
work with them in their own time and space, with the two Micool teachers making themselves available for 
informal training and support for anyone who requested it. It was very much a softly, softly approach and 
willingness to become involved was very much on a voluntary basis. There was a general consensus that this 
was the best approach as teachers did not feel under pressure and could come on board and try out the 
technology and explore its capabilities in a more relaxed way. By the time this research was conducted the 
school had been using the tablets for a full 12 months stretching over two academic years, and seven out of 
35 secondary teachers could be classified as regular iPad users with a number of other teachers using them on 
an occasional basis. Plans were also underway to deliver significant training to all remaining members of 
staff on a whole school basis before the end of June 2017 with a view to increasing the number of regular 
iPad users in the following academic year. 


5. KEY FINDING 3: THE CHALLENGE OF CHANGE 

One of the main motivating factor behind Santo Redentor’s involvement in the Micool project was a 
recognition of the importance of digital technology in the lives of young people and that schools needed to 
change to reflect this. Reflecting on the school’s desire to be involved the school Principal spoke about how 
new technology was changing teachers’ roles and students’ expectations. With so much information now 
available on the internet, teachers were no longer the main source of information for students. In this respect 
one teacher noted how she had started to use resources from American websites because she found that most 
of her students had already visited many of the Portugese websites "and already know the answers to the 
questions I use in class because they have already seen it online The challenge this posed was perhaps best 
summed up by another teacher who noted how the traditional classroom approach no longer engaged students 
as in the past: 

Yes we do need to change. I feel that we are losing students and that is why we always have to try to find 
new things. We cannot get their attention anymore like we did in the past. In the past maybe we didn't get 
their attention, but at least they were quiet. Now they ’re not. And a student that doesn ’t listen and does not 

do things is a student that doesn't learn . This is precisely why we need to change. We have to get their 

attention Because if they are not interested in what they are doing, they don’t work. " 

Teacher A 

According to the school principal the solution to this problem required a change in how schools and 
teachers think and operate - meaning that schools could no longer simply deliver information to students but 
instead had to become more involved in assisting students to become independent learners; this meant 
equipping them with the necessary learning skills to access the vast reservoir of information on the web and 
judiciously using it. Inevitably this would require a change in how teachers planned their lessons and thought 
about their approach to teaching. On a positive note the teachers who were regularly using iPads to support 
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their teaching reported that using the tablets had impacted how they planned for and thought about teaching 
their respective subjects: 

“Yes having the IPad does change how I think about teaching my subject because now instead of just 
delivering content, I can challenge my students to go looking for the content themselves and then we can 
come back later and discuss it together. " 

TeacherE 

“Actually it has diversified methods... I think the work they do now is more how can I put 
it...independent. Not individual. But they work more on their own and it’s not so teacher-based. Do you 
understand? ” 


TeacherD 

While these observations and reflections indicate some important underlying shifts in teacher thinking 
and methodologies among some teachers, it cannot be claimed that this is a widespread development. In fact 
both students and teachers pointed to some very powerful cultural norms and practices inhibiting many 
teachers from embracing changes that new technologies like tablets can bring. Students for example 
expressed the view that they would like to see more of their teachers using mobile devices but felt that some 
teachers had very traditional views about teaching and learning and that this acted as an impediment to 
greater usage. This was further compounded by the fact that they lacked the necessary skills to feel 
competent using such devices: 

7 think some teachers are not ready. They don’t know how..I mean they aren ’t used to computers, they 
are more used to text and so they are old school and more traditional... It’s the way they learned and they 
want to teach like that, more or less. ” 

Focus Group 6 

“It’s seen as a toy and they don’t know how to manage it. It’s hard for them because they learned in a 
traditional way and they don’t have much knowledge about technology and they prefer the board and the 
books. They don’t know how to teach with technology. So they are sort of old fashioned. They need to feel the 
paper and the amount of things you have to study. ’’ 

Focus Group 2 

When asked if they had ever tried encouraging these reluctant teachers to use the iPads by offering their 
expert knowledge in the technology to help these teachers out, the students firmly answered that this is not 
something they had ever considered. Long held traditional views about what constitutes teaching and the fear 
of students knowing more about technology was also cited by teachers themselves as a reason why some of 
their colleagues were not yet using the iPads: 

“Students helping teachers? We never thought about that. I think the reason some teachers avoid using 
tablets is because they don’t want to show that their abilities with technologies are so low. There’s the 
problem. ” 


TeacherD 

Peer observations of teachers using iPads in class as a way of helping reluctant users to become more 
familiar with the devices did not appear to be an option because as one teacher explained “that’s a big 
difficult issues because most teachers don’t want anybody else inside their classrooms. They are very 
strongly against that. ” 

Aside from long held cultural beliefs about teaching norms and practices other systemic and structural 
issues also affected the extent to which tablets were utilised by teachers. The research interviews with 
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teachers revealed how the exam driven nature of the Portugese system meant that teachers felt under constant 
pressure to assess and test out students’ knowledge and abilities. Although teachers wanted to use new 
technologies and devices like iPads to make learning more enjoyable and interactive, the traditional 
examination system that was paper based and knowledge driven, meant that they had to prepare students in 
traditional ways to be ready for such exams. Parental pressure was also a factor, as in the end of the day 
teachers were judged by how well their students passed their school exams in order to secure university 
places: 

“In Portugal we have a huge problem I think because we are a very exam based system. Very> often I 
could do more things but I need more time but I have to deal with all the topics on my curriculum because if I 
don’t and if something comes up on the exams that I haven’t covered I will have parents on my back saying 
“you didn't cover this, they didn’t practice any of this . " Then I cannot tell them, “oh I was trying out new 
methods with the iPads ”. This doesn ’t work. It’s a huge problem and you need your grades to get to 
university’ because otherwise you won’t get a nice place to study 

Teacher A 

Finally the economic crisis of 2008 followed by years of austerity also influenced the extent to which 
some teachers engaged with the iPads. Stringent cuts in teachers’ salaries alongside an increase in teaching 
contact hours from 22 hours pre-austerity to 27 hours post austerity, less annual leave, plus a cessation in 
incentives for career progression whereby teachers who undertook professional development courses in areas 
such as new technology etc. could be promoted to more senior levels, had demotivated teachers. 
Consequently many were now more reluctant to welcome and embrace new developments and invest time in 
trying out new opportunities such as using tablets in teaching. Furthermore, because school numbers had 
fallen by 35% since the recession, the school had not been able to recruit new teachers over that ten year 
period. Consequently a state of stasis had set in with most teachers now in the 40-60 age group, with a 
significant number approaching retirement. This made change and the introduction of new teaching methods 
and new technologies difficult to implement. 


6. CONCLUSION 

This case study has documented and discussed the introduction of iPads in a remote, under-resourced school 
in rural Portugal. Despite a different cultural and linguistic context many of the benefits and attributes 
associated with using mobile technologies in education found in other educational jurisdictions were also 
replicated here, thereby suggesting that the pedagogic value of tablets has a certain universal quality once 
properly deployed. Part of this appeal may be related to the tablets’ versatility and malleability because, as 
this research has illustrated, even teachers working in a very traditional and assessment focused education 
system, found ways of using the device to suit their classroom needs; thereby adding some much-needed 
dynamism and enjoyment to a very rigid and didactic curriculum. The obvious limitation of this research is 
that as a single case study underpinned by a qualitative research methodology it lacks the statistical 
generalizability of quantitative research. Nonetheless, in the tradition of rigorous case study research, it sheds 
a light on some important contextual issues about how innovative projects are adopted and assimilated into 
organisations, including some of the challenges encountered along the way such as how global events like the 
Austerity Crisis had local ramifications in terms of ICT progress and widespread tablet adoption in Santo 
Redentor. 
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ABSTRACT 

This study sought to design and try out a training scheme for college teachers on e-learning use as a classroom strategy in 
a communicative teaching mode. Based on needs analysis the teachers of English were reoriented so that they became 
equipped with the rationale, strategies and assessment techniques of e-learning alongside communicative language 
teaching through the six learning segments which were revised and finalized after a three-day try out. The researcher used 
five stages in the model she used to undertake the study. 

Specifically, the study determined: (1) the stages in the development of the training scheme and (2) the contents of the 
training scheme’s learning segments as a product of the research. 

The current Philippine scenario of college English classes pictures a deterioration of students’ communication skills with 
e-learning left out. Dorothy (2015) says that teaching college English classes has become a chore for many professors. 
Teaching college English is already a particularly trying and frustrating experience. This could be a reason why too many 
college students today are so inarticulate in English that the teacher hardly knows where to begin and how to precede. 
This problem exists in Capitol University and Mindanao University of Science and Technology, the try-out institutes of 
this research. As observed in their English classes, students instinctively switch to the first language whenever they find it 
difficult to express themselves in English - which is particularly most of the time. The researcher noted this during the 
needs analysis stage of this research. 

The scheme produced in this research ensures that teachers will incorporate tasks involving e-learning communication 
mode such as requiring students to do something in the lesson through e- learning, with the language, or with each other. 
Developing students’ oral and written communication skills is one of the most important goals in language teaching. 
These skills are essential for interactive survival in a global setting. The training scheme was designed for this purpose 
thereby incorporating the e-leaming schema. 

The Capitol University professors of English, along with the Mindanao University of Science and Technology 
instructors, have to keep up with the current trends in language teaching. They need to update themselves in terms of 
e-learning use to better achieve the goal of language teaching; which is that of developing students’ oral and written 
communication skills. 

Part of the segments produced by the researcher will also lead these professors to reconstruct their own syllabuses to 
provide for an e-learning communicative class. Howatt (2009) best views the direct application of communicative 
competence to that of a notional syllabus. The notional syllabus includes some description of the grammar of the 
language to be learned (in the form of exponents for the notions and functions) but treats it as just one subsystem of rules 
for realizing a speaker’s ideas, feelings and intentions. This in turn involves another sub-system of different kinds of rules 
(rules of discourse). 

With the use of e-leaming alongside communicative teaching techniques that are contained in the learning segments of 
the training scheme, these professors can provide interaction which will be meaningful, interactive, and responsive to the 
learners’ needs as they teach minor English courses. 
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1. INTRODUCTION 

The relationship between teaching and learning is a complex and fascinating one. In a teaching-learning 
scenario, an interaction that is meaningful, fluid, interactive and responsive to the learners’ needs is ideally 
provided. Hence, English teachers should look for varied ways to respond to the challenge of teaching 
language students. To do this, college English teachers have to be prepared by updating themselves with 
current trends in language teaching such that e-learning is employed. They have to keep up with the latest 
developments in teaching especially that language trends constantly change and higher technology are 
introduced daily. 

Grobler & Bisschof (2006) state that teacher competence is an essential factor for achieving educational 
excellence. To ensure that teachers are accountable and knowledgeable about the subject they teach, 
emphasis has been placed on professional training of new educational systems’ designs. Teachers, as the key 
component of an educational system, need professional training to assure efficiency of students’ learning. 
Thus, educational systems should be driven by the need to achieve efficiency, effectiveness and equity. This 
highlights the importance of training on-the-job teachers. 

For a teacher to develop his competence, the primacy of the classroom is indeed high. It is in that learning 
environment where she/he becomes the principal agent of change. It is where her/his experience is based and 
where growth will take effect (Thomas, 2009). Considering the importance of the classroom in a 
teaching-learning process, this research focused on looking at students in their English classes to see how 
their teachers develop their communicative competence through e- learning. Developing students’ 
communication skills is vital for them to become professionals, Danao (2002) says. She explains in her book. 
Confluence: Journeys that students need to learn the body of material for the profession they are preparing 
for. This body of material in the different disciplines is in English. Most importantly, students need to know 
how to communicate in English since it is an international language, and one of the official languages of the 
Philippines and of Philippine education. Thus, the English subjects in college must equip students to become 
the professionals they want to be, Danao concludes. 

In recent years, much attention has been given to the varying roles that a language teacher has. Increasing 
emphasis has been placed on the less obtrusive roles such as monitoring language use and facilitating 
communication. In some versions of the communicative and task-based approaches there is often no formal 
presentation phase. 

Wayne (2005) states that one of the factors that account for poor oral communication in English among 
students is the failure of many English teachers to provide enough opportunities for oral communication in 
the classroom. In a country like the Philippines, which aims to participate meaningfully in international 
affairs, English has a special place. And so has the English teacher who is called upon to define the role of 
the English language in national development. 

However, it has long been known that teaching does not necessarily equal learning - that what a teacher 
does in the classroom to teach may not match what the learner perceives the lesson to be about. Prestia 
(2013) claims that the same teaching method does not work for all learners for they learn in different ways 
and that teachers should employ a variety of methods in teaching, learning and assessment. The effect of 
what learners learn in every learning episode may vary, depending on how well they are motivated and how 
ready they are to absorb presented ideas. 

Davis (2003) states that the key to teaching students to think lies in how the teaching process is 
conducted. Teachers have the command to raise the level of students’ thinking even to the extent of analysis 
and appreciation. Since students do not think this way naturally, interaction is necessary. High quality class 
interaction contributes significantly to a student’s progress in developing his communication/verbal skills. 
This in turn will lead him to develop his thinking skills. Hence, it can be deduced that it is indeed a necessity 
to develop students’ communicative competence, one of the goals of this research. 
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2. METHODOLOGY 

The Research and Develop (R&D) method was used in the study to gather information about the college 
English classes of the two try out institutions. Seels and Glasgow’s (1995) ADDIE model was modified by 
the researcher into Needs Analysis, Plan, Create, Try Out and Assess (NAPCTA) and was followed 
throughout the duration of the research with the incorporation of the e-learning strategies. 

In the needs analysis stage, baseline data were obtained from the respondents through the use of Focus 
Group Discussion (FGD) to gather the type of teaching strategies used in college English classes; the 
questionnaire for identifying the lesson objectives; the classroom observation which looked into the three 
parts of the lesson; and the analysis of the content of English tests. From these results, the researcher came up 
with a training scheme composed of six learning segments for communicative teaching in college English 
classes. 

The previous citations, as presented, defeat the goal of language teaching - that of developing students’ 
oral and written communication skills. College English teachers then must try to hold on against the 
precipitate erosion of the position of English in the Philippines. This can best be fortified at the plane of the 
teaching of English in college. College professors have to keep up with the current trends in language 
teaching by adopting e-learning strategies in classroom teaching. 


3. RESULTS AND DISCUSSIONS 

Making use of the four extensive needs analysis procedures; the data were then analyzed. These were 
analyzed individually and entered in a matrix in the form of focus, rating and description. Final evaluation 
was derived through these three categories. The matrix showed commonality of results through simple 
frequency counts. Results served as basis for the production of the training scheme’s learning segments. 

To illustrate, the data gathered appeared in a matrix form as shown on the next page. 

Table 1. Needs Analysis Results: A Summary 


Instrument 

Variables Measured 

Description 

Focus Group Discussion 
(FGD) 

Teaching Strategy 

Communicative/ 
E-learning slightly used 

Questionnaire 

Lesson Objectives 

Non- 

Communicative/E-learning 
in maximum use 

Class Observation 

Strategy, Content, Evaluation 

Non- 

Communicative/E-learning 
not used 

Test Analysis 

Tests, Textbooks, Work texts 

Non- 

Communicative/E-learning 
used in maximum 
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As shown in the above matrix, there is a need to reorient English teachers’ perspectives toward 
communicative teaching. A training scheme was designed to cater to this need. 

Three experts then evaluated the designed training scheme composed of six learning segments. The table 
on the next page is a summary of the checklist for assessing the learning segments. The three respective 
trainers on a scale of 1 -5 rated these. The scale is as follows: 


Very much - 5 

Much - 4 

Just enough - 3 

Not much - 2 

Not at all - 1 


Table 2. Trainers’ Assessment of the Learning Segments: A Summative Checklist 


ITEM 

T 1 

T2 

T3 

A. FORMAT 




1. General Appearance 




Is the material likely to appeal to the user’s aesthetic sense? 

5 

5 

3 

2. Component 




Does the material contain many components that the trainees will have 
difficulty keeping track of them? 

3 

2 

4 

3. Quality 




Did the researcher use high quality materials in the production process? 

5 

5 

4 

4. Appropriateness of Illustrations 




Are the illustrations of the materials appropriate to the activities? 

4 

4 

3 

5. Readability of the Material 




Is the material readable? 

4 

5 

4 

B. ORGANIZATION AND CONTENT 




6. Approach 




Does the researcher use an approach consistent with the prescribed topic? 

4 

5 

3 

7. Instructional Objectives 




Are the objectives compatible with the ones prescribed in the segment title? 

5 

4 

3 

8. Scope and Sequence 




Is the scope and sequence of the material compatible with the time frame of 
the training? 

4 

4 

3 


4. FINDINGS 

After the results of the study were gathered, the following relevant points were seen: 

1. The college English teachers reported that they developed students’ communication skills through 
question and answer activities in an e-learning mode. These were predominantly observed in the 
initial stage of this research. Most teachers engaged the students in the lesson by letting them recite 
only when called. Students also did picture talk. This was seen as another technique used to 
encourage student talk. In this activity, students formulated sentences out of an exposed picture 
using the present progressive tense. 
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To promote students’ participation in class, the college English teachers used group dynamics in an 
e-learning setting. Belonging to a group, each student answers questions orally and earned points. 
They were not allowed to sit unless the question was answered. 

The latter activity served as an evaluative measure in the class as observed. It also served as an oral 
exam. 

2. The results of the needs analysis showed that teachers needed to develop competencies in 
communicative teaching, thereby making maximum usage of the e-learning software. A training 
scheme needed to be designed for this purpose. Through the needs analysis, the following needed 
competencies of teachers were discovered. Firstly, teachers are required to value the affective side 
of language learning. These teachers believed that students perceive language learning negatively 
since they observed that students either felt uneasy or tensed when asked to speak English. Also, 
teachers believed that they could not be blamed for these students who use un-English utterances. 
When they come to college they are already made, a teacher commented. 

Moreover, teachers need an update on the use of e-learning classroom strategies through 
communicative teaching. The lecture method, one that is teacher-oriented was predominantly done 
in English classes. Very limited chances allowed for student talk. More importantly, teachers have 
to be exposed to the different strategies of communicative teaching. The Q and A technique was 
overused. If only they can observe a strategy that really works, they will certainly use it in class. 
This is one of the reasons why the training scheme was designed. 

Another avenue that teachers obviously need to acquire is to examine their course titles. By then, 
they will see what topics fit in the descriptive title of the course. There were several topics seen in 
the syllabus which twisted the idea given in the course title. English 1A for instance, a study and 
thinking skills course, focused solely on the parts of speech for its topics. 

Very common evaluative measure used to test students’ skills was the paper and pencil test. 
Teachers are believed to be more creative than adopting just one mind-numbing strategy. Therefore, 
they need exposure in communicative testing and assessment. By then, not only quizzes and tests 
will occur in English classes. 

3. To develop a training scheme for communicative teaching, the following stages were followed: 
Needs Analysis, Plan, Create, Try Out, and Assessment (NAPTCA) model. The analysis stage 
which made use of FGD, classroom observation, questionnaire and documentary evaluation of tests 
gave the researcher sufficient information on what competencies are needed by college English 
teachers to develop communication skills of students. The training scheme was then designed based 
on the needed competencies of English teachers. 

The development stage of the scheme had for its baseline data the needs analysis. The scheme, in 
the form of segments, was then produced. Each segment contained a topic virtual to communicative 
teaching. Then, the implementation stage followed, actualized in the training of the college English 
teachers of Capitol University. The trainers who implemented the scheme were experts in the field 
of communicative teaching. 

Before the try out the trainers did a close examination of the segments. They took part in the 
development of the scheme since they brought materials for sample activities and valuable input. 
After the try out, the refinement of the scheme was done. This was based on the evaluation made by 
the trainers and trainees. During the training, the trainees evaluated the segments in the huddle 
sessions. The segment content and trainers’ presentation of the learning segments were assessed. 
After the training, evaluation sheets were given to the participants of the training. They rated the 
totality of the scheme using the modified evaluation scheme. The comments served as basis for the 
revision of the learning segments. 
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5. CONCLUSIONS 

Taking the findings as strong points for evaluating this research, the following conclusions were drawn: 

1. Various opportunities for e-learning talk and student-talk should be provided in English classes. 
This calls for teachers’ creativity in designing meaningful and communicative tasks. After all, 
communication is but the goal of language instruction. Hence, the usual Q and A technique may not 
be overused, as there are other strategies to choose from. 

2. The needs analysis showed that English teachers lack competencies with the use of E- learning. The 
activities provided in class did not encourage class interaction. They were the correct usage type if 
not rote memorizing of the rules of grammar. This therefore, made students more conscious to 
speak, as they had to think of the correct usage of verbs all the time. 

3. Instruments such as FGD and questionnaire cannot be solely relied upon as baseline data in 
research. Some noted information did not actually happen in the classes observed. This shows that 
teachers were aware of the idea of e-learning and communicative teaching but did not apply this in 
class. 

6. RECOMMENDATIONS 

The following concepts are recommended as a result of the study: 

1. An experimental research may be conducted to test the effectiveness of the developed training 
scheme. This may be compared to another scheme, which gives focus on the grammar approach to 
teaching English utilizing the e-learning scheme. 

2. Teachers need to attend trainings with the design showing the apportioning of the topics per day, 
with the assessment and activities built into topics. Hence, a designer should detail a training 
scheme so that a trainee who uses it will just follow using the e-learning serration data schema. 

3. Trainings should be conducted on days other than Monday. There is such a thing as blue Monday. 
People will be late, still unprepared because of the weekend’s activities. 

4. Too long content of segments may bore the readers/trainees. As such the presentations of concepts 
have to be capsulated in various formats. 
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ABSTRACT 

There is a shortage of STEM (Science, Technology, Engineering and Mathematics) educated workforce in the US, 
especially among minority and underrepresented groups. Recruiting and retaining STEM majors has been a major 
problem for universities and community colleges for many years. The Computer Science department of University of 
Houston-Clear Lake (UHCL) has been collaborating with three campuses of San Jacinto College (SJC) on recruiting and 
retention strategies. Our activities started taking off after we received a joint state grant to support our recruiting and 
retention activities. The grant allowed us to experiment with several pilot projects such as peer-mentoring and 
programming challenge. We then expanded our pilot project to include additional STEM fields and additional recruiting 
and retention strategies. The additional strategies included monthly Tech Fridays, annual Extended Orientation and 
annual STEM Challenge. We submitted a proposal to the US National Science Foundation (NSF) and received a 5-year 
joint grant from the STEP program of NSF with the start date of September 2013. Our NSF grant activities have been 
very successful. In 2016, we expanded our activities again by adding additional elements such as undergraduate research, 
internships, and creation of a new computer science teacher certification program. We submitted a proposal to the 
HSI-STEM program of the US Department of Education (DOE) with a focus on recruiting and retaining students from 
Hispanic or low-income families. We received a 5-year grant from DOE with a start date of October 2016. In this paper, 
we share our strategies and experience with conference participants, hoping that our experience will help others who are 
considering similar strategies for recruiting and retention of STEM majors. 
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1. INTRODUCTION 

In the United States (US), the number of occupations in STEM (Science, Technology, Engineering and 
Mathematics) fields has been growing rapidly in the past 10 years and the projection is that this growth rate 
will continue in the future [Noonan 2017]. The US lacks enough educated workforce to fill existing open 
STEM positions [Laros 2016]. Recruiting and retaining STEM majors has been a major problem for 
universities and community colleges in the US for many years. Until fall of 2014, our university, the 
university of Houston-Clear Lake (UHCL), was an upper-level institution. We offered only 
junior-, senior- and graduate-level courses. We did not offer freshman- and sophomore-level courses. Our 
undergraduate students mainly came from a handful of surrounding community colleges. In order to increase 
our undergraduate enrollment in STEM fields we had to help our main feeder colleges to fill their STEM 
classes. Ten years ago, we started collaborating with three campuses of the San Jacinto College (SJC) on 
recruiting and retention activities. At the time, over 50% of the UHCL undergraduate students came from 
SJC campuses. Our collaborative activities started taking off after we received our first joint grant from the 
Texas Workforce Commission. The title of this state-supported grant was “Computer Science Scholars: 
Recruiting, Retention, and Mentoring“ [Abeysekera 2008, Davari 2007, Davari 2009]. The grant allowed us 
to experiment with several pilot projects. The results of our pilot projects were promising. We then started 
working on a new proposal to get a longer-term grant to be able to expand on our pilot projects and to include 
additional STEM fields. We submitted the proposal to the US National Science Foundation (NSF) for 
funding. On our third try, we received a 5-year joint grant from the STEP program of NSF with the start date 
of September 2013. The title of this grant is “Bridges to STEM Careers (BSC)”. Our NSF grant activities 
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have progressed very well [Abeysekera 2017, Perkins-Hall, 2017, Rosales, 2016], In 2016, we expanded our 
activities again and prepared a new proposal for submission to the HSI-STEM program of the US 
Department of Education (DOE) with a focus on students from Hispanic and low-income families. We 
received a 5-year grant from DOE with a start date of October 2016. The title of this grant is “Pathways to 
STEM Careers: A University-Community College-Industry Collaboration.” In the following sections, we will 
expand on our major project activities in each of the three funded projects in chronological order. 


2. DETAILS OF THE THREE FUNDED PROJECTS’ ACTIVITIES 

2.1 Computer Science Scholars: Recruiting, Retention, and Mentoring 

In 2006, we had a scholarship endowment grant for Computer Science (CS) majors from the state of Texas 
and a NSF Course Curriculum and Laboratory Improvement (CCLI) scholarship grant. These grants were 
producing a large number of scholarships each year. However, we could not find enough qualified candidates 
for these scholarships among the student population of local community colleges. We were also witnessing 
large drop rates on our campus in classes that transfer students took. We therefore decided to combine our 
recruitment and retention efforts with our SJC colleagues, in order to increase the number of CS majors and 
to improve the quality of the CS programs at all our campuses. We submitted a joint grant proposal to Texas 
Workforce Commission and received a two-phase grant with a total amount of $300,000. Our proposed 
strategies included joint recruiting and retention strategies through peer mentoring. For recruiting, we 
prepared recruiting materials that highlighted the importance of majoring in CS and the benefits of joining 
SJC and then transferring to UHCL. We reached out to high school counselors and teachers, and arranged 
class visits. For retention, we created peer-mentoring centers in both institutions supported by eight 
undergraduate and one graduate CS major students. These qualified students mentored their peers in all 
programming- intensive courses that the incoming students took in their first year at each campus. 

Student mentors received salaries paid by the grant that supplemented their scholarships. Even more 
rewarding was the reinforcement of the student’s knowledge in computer science courses through their 
mentoring activities. Student mentors also participated in the preparation of the recruiting material for 
on-campus and off-campus events. In the second year of this grant, with the help of student mentors, we 
planned and executed a successful programming challenge. Seven of the winners of the challenge joined our 
CS program. After the success of these pilot projects, we decided to prepare proposals to receive longer-term 
federal grants to allow us to continue our efforts, to expand on our strategies, and to include more STEM 
majors. 

2.2 Bridges to STEM Careers (BSC) 

After two unsuccessful tries with NSF programs, we received a 5-year STEM grant from the NSF STEP 
program for our expanded project. The total amount of this grant was approximately $1.5 million. The main 
goal of this project was to increase the number of students ultimately graduating with STEM degrees. As 
such, our goals included an increase in the number of students graduating in STEM fields from each of the 
three campuses of SJC (a two-year college), facilitate and increase transfer of STEM majors from SJC to 
UHCL, and an increase in the number of students graduating in STEM fields from UHCL. We had proposed 
specific numerical goals for each of these sub-goals. We also intended to increase student engagement and 
persistence through the development of face-to-face and technology-enhanced learning communities, and 
collaborate with local industry professionals. The STEM fields we focused on included: Computer Science, 
Computer Information Systems, Information Technology, Computer Engineering, Mathematics, and Physics. 
Our supporting initiatives included an extended orientation during summer before the start of fall semester, 
peer-mentoring and tutoring sessions, development of a cyber-center to archive and disseminate resources, 
monthly Tech Fridays, annual STEM challenge with participation and sponsorship from local industry, and 
modest financial aid supporting students in STEM majors. Peer-mentoring and hands-on experiences with 
technology have shown to encourage students to choose and succeed in STEM fields [Jolly 2004, Zhao 
2006], 
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Assessment by the external evaluator of our project has shown that students who have participated in 
these activities have gained confidence to continue in their STEM education. Participating students identified 
with other students with whom they can relate. We intend to provide a sustainable model that increases 
retention rates and provides opportunities for students to succeed in STEM fields. 

All our project elements are based on best practices reported in the literature [ASE report 2012] and 
successful pilot studies [Abeysekera 2008, Davari 2007, Davari, 2008]. Student participants of the project are 
given opportunities to get directly involved in the design and the development of the project’s cyber-center. 
Students are provided opportunities to practice teamwork, obtain internships, and have exposure to emerging 
technologies such as robotics, mobile application development, computer forensics, cyber-security, computer 
game programming, web design, video editing, etc. 

The BSC Executive team, comprised of the Primary Investigator (PI) and co-PIs from all campuses, 
meets each month to plan and coordinate project activities. A main goal for student activities is to make 
students from other campuses feel welcomed, particularly those from community colleges, so that they might 
be more inclined to attend UHCL and complete their bachelor’s degrees. The BAS project activities are given 
below. 

2.2.1 BSC Club 

A BSC club is active at all four campuses. Club members are encouraged to participate in activities and 
propose new activity topics at any of the campuses. Students use the clubs as an opportunity to network with 
other students and faculty. 

2.2.2 Peer-Mentors 

Each campus has a team of students who have various responsibilities, including tutoring; planning and 
executing BSC club activities; planning and executing monthly Tech Fridays; and preparing and 
administering the annual STEM challenge. Student mentors at all campuses attend targeted classes and serve 
as in-class mentors [Perkins-Hall 2017]. Mentors also hold regular office hours and act as tutors. They hold 
weekly recitation sessions on key topics in computing, engineering and mathematics. There were 13 different 
peer mentors throughout the year last year at UHCL. 

2.2.3 Tech Fridays 

Tech Fridays are designed by mentors and faculty to create learning opportunities. Industry and academic 
speakers are invited to provide relevance. Tech Fridays are held for three hours and consist of hands-on 
activities, worksheets and interaction with topic experts. 

Tech Friday topics to date have included: 

• Robotics: students work with Arduino boards and sensors. 

• Computer Forensic Investigation: students worked on solving a simulated case using forensic 
software tools. A Houston police officer talked with students about his experience as computer 
forensics specialist and the opportunities in the field. 

• Web Development: students are taught how to develop web pages. 

• Video Editing: students learn how to edit sound, image, and video using free software. 

• Arduino and Charlieplexing: students wire LEDs and write programs to generate light sequencing. 

• Computer Game Programming: students develop computer games using free software. 

• Sound-to-Light: students wire kits to convert sound to light. 

• 3D Printing: arrangements are made with a local library to utilize their 3D printer for training and 
practice. 

• Soldering: this also takes place at the local library. We provide the soldering kits. 

• Python Warrior: this Tech Friday introduces Python Warrior to create a simple graphical game in 
which a player moves through levels avoiding enemies, gaining health, and attacking enemies. 

• Swarmathon: the Robotic Operating Systems (ROS) is presented and entry-level applications on 
robotics are developed. 

The events are usually at capacity with 35 to 50 students attending each event. 
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2.2.4 STEM Challenge 

Each spring semester teams of students are invited to UHCL for a STEM challenge. The challenge requires 
teams to participate in three stations and complete several tasks of increased difficulty in succession. We 
categorize teams into beginner level or advanced level. The events are timed and checked for completion. 

Challenges are open to high school and undergraduate students. Participants come from local high 
schools, home-schools, community colleges, universities, and sometimes middle schools. The first year was a 
robotics challenge and the three stations consisted of: 

• Controlling Arduino and lights to produce Morse codes 

• Reading sensor data such as temperature and water 

• Moving a robotic tank through an obstacle course. 

In subsequent years the challenges were renamed STEM Challenge with three stations that consisted of: 

• Game of Clues to challenge math knowledge 

• Creative Inventors to showcase imaginative skills 

• Robot Adventure to test problem-solving and programming skills 

Winning teams are offered scholarships to UHCL, summer internships with local industry, and other 
prizes. Raffle prizes are given to every participant. 

Local industries and businesses sponsor the event and provide prizes. Industry partners and faculty serve 
as judges and technical program supervisors. 

The events have always been at capacity. The robotics challenge hosted 15 teams and the latest STEM 
challenge hosted 29 teams made up of 111 students participating in the competition at the beginner and 
advanced levels. For more details about STEM Challenge, please refer to a companion paper titled 
“Challenges for a New Generation of STEM Students” to be presented in this conference. 

2.2.5 Extended Orientation 

We prepare and hold a 2-day extended orientation in August of each year. The orientation brings together 
students from various stages of their STEM education. On the first day, SJC students convene at one of the 
SJC campuses and new UHCL STEM majors convene at UHCL. At SJC, students are introduced to the 
campus, to student leaders and to faculty. A motivational speaker also presents at SJC. At UHCL, students 
participate in activities to get familiar with the campus. They also engage with a faculty panel and a student 
mentor panel, and are assigned to specific BSC mentors to guide them throughout the academic year. On the 
second day, student participants from all 4 campuses convene at UHCL for Team Building Activities. They 
engage with a panel of academic and transfer advisers, financial aid officers, and student services 
representatives. 

In the most recent orientation, each day was an afternoon event that commenced with lunch at noon. 56 
students attended the 2016 orientation. Each student received a BSC binder, a pen and a BSC t-shirt. 

2.2.6 Cyber Center Website 

The current functionalities that the Cyber Center hosts include: information about the BSC project, a news 
and comments system, site membership, events’ schedule and registration, photo and video archives from 
past STEM events, and other resources. UHCL hired a research assistant, paid by the grant, with the primary 
responsibility for developing and maintaining the Cyber Center. The planned additional functionalities 
include increased interactivity, more useful resources, and social media integration. 

2.2.7 Internal/External Advisory Board Meetings 

The Internal Advisory Board (IAB) for the BSC project consists of UHCL and SJC administrators and senior 
faculty members. The External Advisory Board (EAB) members consist of members of local STEM 
industries. We have been meeting with the IAB semi-annually, and with the EAB annually to report progress 
and to receive feedback and advice. In spring semesters, we combine IAB meeting and EAB meeting. In this 
meeting, we report to the board highlights of our major grant activities in the preceding year. This venue has 
given BSC Executive team the opportunity to learn of ways to improve the efficiency of our project 
activities, such as continuing with the theme of the annual STEM challenge, incorporating more hands-on 
activities, and involving outside experts in project activities. 
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2.2.8 External Evaluator 

The BSC project has an external evaluator who evaluates the effectiveness of major project activities. The 
evaluator collects a mix of qualitative and quantitative data. The qualitative data is collected via face-to-face 
interviews with students, open-ended survey questions, and observations during the events. The quantitative 
data is collected via pre- and post- surveys that measure students’ preparedness, self-perceived aptitude, and 
attitudes/interest pertaining to STEM. Pre- and post-surveys are generated via Survey Monkey. A link to the 
survey is provided to each student and students access the link using cell phones or laptops. Students may 
also opt to take a hard copy of the survey. The evaluator stays in compliance with all UHCL IRB restrictions. 
Survey data is analyzed using SPSS. The results are provided to the BSC Executive team for possible actions. 

2.2.9 Significant Results 

Last year 228 unique students (50 female and 169 male) participated in one or more BSC project events such 
as Tech Fridays, STEM Challenge, and Extended Orientation. These are students who registered and 
participated in these events. It does not include the number of students who were mentored in classes, or 
those who were mentored in the mentoring centers or those who participated in recitation sessions. At UHCL, 
the total number of students in CS1 and CS2 classes in fall 16 and spring 17 semesters was 133. Many of 
these students received one-on-one mentoring in class and in the mentoring center. Students who visited the 
mentoring center and participated in recitation sessions also came from other classes in Computer Science, 
Computer Engineering, and Mathematics. 

The enrollment data in the six STEM programs that are the main focus of the BSC project are provided in 
Table 1 below for spring 2013, which was the semester before the BSC project started, and for the spring 
2017: 


Table 1. Total Enrollment Data 


Program 

Enrollment in Spring 13 

Enrollment in Spring 17 

Computer Engineering 

37 

146 

Computer Information Systems 

58 

98 

Computer Science 

51 

185 

Information Technology 

51 

74 

Mathematical Sciences 

51 

127 

Physics 

15 

36 

Total 

283 

666 


We should note that starting from fall 2014, UHCL switched from an upper-level institution to a 4-year 
institution. Therefore, in addition to the students who transferred from area community colleges, we have 
also been recruiting freshmen from area high schools starting from fall 2014. In Table 2, we show the change 
in the number of students who have transferred from the three campuses of SJC. The data shows that the total 
number of transfers from SJC campuses in the focused STEM programs increased from 153 in spring 2013 to 
281 in spring 2017, an increase of 128 students (over 83%). 


Table 2. Transfer Enrollment 


Program 

Transfers in Spring 13 

Transfers in Spring 17 

Computer Engineering 

21 

54 

Computer Information Systems 

36 

42 

Computer Science 

33 

73 

Information Technology 

23 

46 

Mathematical Sciences 

30 

51 

Physics 

10 

15 

Total Transfer 

153 

281 


In addition, we have established strong relationships and benefitted from the involvement of our 
community/industrial partners. Representatives of these organizations have continued to give valuable input, 
help with the STEM Challenge, and attend our External Advisory Board meetings. This grant has allowed us 
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to enhance relationships with colleagues from community colleges and high schools, and strengthen 
relationships among students from all campuses. 

We have witnessed significant professional growth from our student mentors. They develop ideas for 
Tech Fridays and present them. This year, three peer-mentors after graduating with their bachelor degree 
were accepted into doctoral programs at other universities. Other peer-mentors who have graduated have 
found lucrative jobs in local industry. Mentors have come from a diverse group, including Whites Americans, 
African Americans, Hispanic, Asian and women. 

Students have indicated that their participation in BSC activities has made their transition to our 
university easier. Students become familiar with UHCL before they enroll here. Transfer students know 
UHCL students and faculty. They have the opportunity to get their questions answered by peer mentors and 
faculty, and benefit from early advising. 

2.3 Pathways to STEM Careers: A University-Community College-Industry 
Partnership (PSC) 

In fall of 2015, one year after UHCL started offering freshman and sophomore classes, we had a Friday 
Morning Breakfast meeting with teachers and counselors from area high schools and community colleges. 
Our goal in this meeting was to find out if their students could easily transition to our STEM programs. 
Several new ideas came out of this morning conversations, which we were not addressing in our existing 
STEM project activities. Afterward, jointly with our colleagues from the UHCL College of Education, we 
prepared a grant proposal to address the main issues that were raised in our meeting with folks from high 
schools and community colleges. Many of these high schools and colleges have the Hispanic Serving 
Institutions (HSI) designation. We submitted the proposal to the HSI-STEM program of DOE. We received a 
5-year grant from DOE with a total amount of over $3.77 million and a start date of October 1, 2016. 
Unfortunately, our community college partners with whom we have been collaborating could not join us in 
this proposal because they had lost their HSI designations. 

2.3.1 Major Goals of PSC Project 

(a) Increase retention rate and the number of graduates in STEM fields among Hispanic and 
low-income students, by means of peer-mentoring, tutoring, and counseling; 

(b) Develop model transfer and articulation agreements in STEM fields with area HSI colleges; 

(c) Develop Computer Science 8-12 Teacher Certification Degree Program; 

(d) Provide culturally responsive professional development opportunities for faculty of UHCL, 
faculty in area HSI colleges, and area STEM teachers; 

(e) Provide opportunities for Hispanic/low income students to conduct paid scientific research 
with UHCL faculty; 

(f) Provide paid internship opportunities with local industry for Hispanic/low income students. 

We have started the preliminary work on this project. The grant supports a full-time program director for 

5 years. We also have the budget for 10 student peer-mentors’ positions, 8 research assistants’ positions for 
conducting research under the supervision of our faculty, and 8 paid internship positions to place recipients in 
positions at local industry and high schools. We conducted the search and hired a well-qualified program 
director. With the help of the program director, we are identifying qualifying cohort students, interns, 
research assistants, and peer-mentors. An applicant to any of these positions must either be Hispanic or be 
qualify as a low-income student. 


3. CONCLUSION 

Collaboration with the participating colleges has strengthened relationships between students, faculty and 
administration within all campuses. It has established a sense of community among students, faculty and 
industry partners. Students will make time to attend and participate if there is an opportunity to learn and 
experiment with STEM topics. The hands-on STEM activities will get student participants excited about 
pursuing careers in STEM. Tech Fridays also provide students an opportunity to build relationships with 
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other students interested in STEM careers, and gives them a chance to meet faculty and students from other 
campuses. STEM Challenge makes our students want to learn more about computer hardware, robotics and 
programming. When asked what was most liked about Tech Fridays, the responses included “The hands on 
working”; “That I got deep into how the computers actually work to access information. I learned tons of 
things 1 never knew about. I am definitely coming back next time.”; “This was an enjoyable experience. The 
infusion of mathematics was nice. Students need to know number theory!”; “I liked the relaxed, fun 
atmosphere.” 
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ABSTRACT 

The concepts of future classrooms, multimedia labs or active learning space has recently gained prominence in 
educational research. Evidence-based research has found that well-designed primary school classrooms can boost 
students’ learning. Also, schools’ principals, teachers and students are requesting for more flexible, reconfigurable and 
modern classrooms’ layouts, where technology and active pedagogical practices can be incorporated into an easier way. 
Under the scope of TEL@FTELab Project (Technology enhanced learning at Future Teacher Education Lab) of the 
Institute of Education of University of Lisbon an empirical study was conducted with 82 teachers of elementary and 
secondary schools aiming to capture their vision about what the classrooms for the future should be. Data was collected 
through a focus-group methodology. Teachers were asked to form groups of 3-to-8 elements and challenged to build a 3D 
mock-up of their future classroom by using a 1:20 scale kit provided by the researchers. The process of the classrooms 
construction was videotaped and content analysis of the mock-ups was conducted. This article presents the results of the 
data collected, focusing specifically in the following aspects: descriptive key concepts of what is seen as a future 
classroom, spatial organization (different working zones identified by the teachers), physical elements (furniture and 
equipment) and environmental aspects (light, sound, air quality, temperature, colour, natural elements, comfort and 
security). 


KEYWORDS 

Future Classroom, Learning Spaces, Pedagogical Innovation, Teacher Education 


1. INTRODUCTION 

The concepts of future classrooms, innovative multimedia labs, active learning spaces or next-gen schools 
has recently gained prominence in educational research. The number of studies in this field started to grow, 
yet most of them focus on higher education institutions, scarcely contemplating other education levels, and 
are related to the assessment of buildings renovation projects that did not consider the needs and wishes of 
the main stakeholders: teachers and students. This study aims to collect inputs from what one of these 
stakeholders - teachers - identified as relevant for the development of the classrooms of the future for 
elementary and secondary schools. 

Considering school buildings external factors, several authors have identified their impact on multiple 
human functions, including cognitive processes (Hygge & Knez, 2001) and well-being (Evans, 2003). 
Higgins et al. (2005) emphasize the significant impact that elements such as temperature, luminosity and 
acoustics have on school's internal environment. Montazami, Gaterell, and Nicol (2015) concluded that 
students and teachers' performance is influenced by the internal environment of buildings, specifically by 
factors such as noise levels, indoor temperature, air quality and lighting. These factors are positively 
correlated with students' learning and behaviour (Guardino & Fullerton, 2010), as well as their satisfaction 
(Butt, 2010; Hill & Epps, 2010) and academic performance (Barrett, Zhang, Davies & Barrett, 2015; Mendell 
& Heath, 2005; Samani, 2012). Through the years and very consistently, literature has been providing clear 
evidence of the effect that the overall design of the physical environment of learning spaces has on its users. 
More recently, studies have also alert to the privative impact that long periods of inactivity have on pupils’ 
body health, as well as on its cognitive development and behaviour control (Hillman et al., 2014; Kilbourne, 
Scott-Webber & Kapitula, 2017). However, Imms and Byers (2017) advocate they the literature around this 
topic is still scarce and that this topic claims for more deep and robust research. 
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2. RESEARCH SCOPE AND METHOD 

This paper presents the data collected under the scope of the Technology-enhanced Learning at Future 
Teacher Education Lab (TEL@FTELab) project. This R&D (research and development) project aims to 
develop knowledge that fills the need of powerful engaging strategies to support the development and 
adoption of innovative teacher education practices that can empower future teachers to efficiently act as 
educators of the succeeding generations. This three years project is organized in three phases. The Phase I is 
concerned with the design and setup of a Future Teacher Education Lab, a prototype of a future classroom for 
teacher training, as well as training modules developed in alignment with a 3D 21st century teacher skills 
framework. The Phase II focuses on piloting the training modules developed in phase I, in two consecutive 
cycles of implementation within 4 Master Programs on Teaching (Biology, Informatics, Mathematics and 
Physics). Each cycle of piloting includes the co-design of learning scenarios between teacher educators and 
student teachers and its experimentation in real secondary school classes of Biology, Informatics, 
Mathematics and Physics. Finally, the Phase III takes the data collected from the pilots conducted on phase II 
and produces a set of video cases, reviewed training modules and learning scenarios that together with the 3D 
framework compose a Teacher Education Toolkit, which aims to be the main outcome of the project. 

The data under analysis was collected in the project Work Package 2 which aim to design the 
architectural space of an innovative classroom for teacher education, named Future Teacher Education Lab, a 
reconfigurable classroom organized into different working zones, built for promoting the development of 
different learning and teaching practices (more information available at http://ftelab.ie.ulisboa.pt/#fte-lab). To 
do so, an empirical study was conducted for collecting inputs of what such a classroom should be from 
different stakeholders: architects, designers, teacher educators, in-service teachers, future teachers and 
undergraduate students. 

2.1 Participants 

This paper addresses, specifically, the data collected from in-service teachers. 82 elementary and secondary 
school teachers took part of the empirical study; 59 women and 23 men. Teachers age range was between 26 
and 52 years old. Aiming to capture their vision about what the classrooms for the future should be, a 
focus-group interview was adopted and 16 groups were formed, between March 2016 and July 2017. The 
group numbers range from 3 to 8 participants. 

2.2 Procedures 

Teachers were invited to participate in a Participatory Design (PD) process. This is a holistic research 
approach frequently used in the domains of design, architecture and urbanism. PD is an approach to design 
that attempts to actively involve the potential users in the design process to help ensure that the designed 
product/service meets the users’ needs (Sanders, 2002). To support this process a 3D toolkit was created to 
promote a creative act of designing a future classroom 3D model. The manipulation of the toolkit gives the 
participants the ability to express their own specific ideas and to put them in practice by build a physical 
future classroom model. The toolkit was composed by a set of images, words, icons and symbolic pieces 
selected from data collected in a previous stage of the TEL@FTELab project (Pedro et al., 2017): 

. 3 principles (Pedagogy, Technology and Space) and 15 keywords (Innovation, Creativity, Dynamic, 
Collective, Autonomy, Inclusion, Flexibility, Collaboration, Engagement, Feedback, Multiplicity, 
Interactivity, Communication. Personalization and Equality), even though other principles and keywords 
could be added by the participants; 

. Colourful cardboards representing different types of learning activities/zones: Individual activities 
(beige). Group activities (blue); Technological activities (yellow). Ideas creation/brainstorming activities 
(purple). Projection/multimedia activities (green), 3D printing and digitalizing activities (red). Informal 
activities (orange) and ‘to-be-defined’ activities (white); 

. Printed symbols representing environmental properties: Trees, Gardens, Sun, Food, Ventilation, 
Temperature, Humidity, Comfort, Air, Acoustics; 

. 5 printed symbols and 9 words representing educational technology and digital devices; 

. 26 classrooms related pictures; 
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. 3D blocks representing teacher (1) and students (1), windows (2), door (1); glass walls (2) and 
10 wooden small boards representing furniture (tables, chairs, cupboards, puffs, room dividers); 

. 6 emotions related icons: 3 smiles (representing like, dislike and neutral expressions) and 3 colours (red, 
yellow and green representing respectively bad, medium and good conditions). 

All the physical elements were 1:20 scale; the toolkit also included one polystyrene board 0,50x0,50m 
(10x10 m=100m 2 ) that represents the classroom floor. 

A session for data collection was organized upon a focus-group methodology. In the first part of the 
session, participants were asked to select the key concepts that best described their vision of what a ‘future 
classroom’ should be. They had to select one out of three principles and five out of fifteen words. Participants 
had 15-to-20 minutes to discuss and choose these concepts. In the second part of the session, participants 
were provided with a 3D toolkit and were asked to build up their future classroom, grounding their design 
and layout choices upon the selected key concepts. 20-to-25 minutes were provided to this task. Additionally, 
at the end, each group were asked to make a 5-minutes presentation, explaining their ‘future classroom’ 
mock-up. 

Being aware of the constraints that were inherent to the provision of a toolkit (with a fixed set of images, 
words, icons and symbolic pieces), participants were provided with extra paper, colour markers, scissors and 
glue stick for creating any other type of elements that were seen as relevant but not findable in the toolkit. 
The instructions given by the research team emphasized that this activity should be seen as a free and creative 
act; participants should be involved in an open discussion process and this should result in the design of a 
future classroom 3D model. 

The main goals of this session were twofold: (1) to understand participants’ underpinning principles and 
concepts when conceiving and planning a future classroom environment; and (2) to comprehend how 
participants materialized these concepts into classroom teaching and learning spaces, by using the toolkit. 

The following images show the edification process of the future classrooms, as well as an example of the 
outcomes of this data collection process. 



Figure 1. Pictures of the Future Classrooms 3D Models: Process of Construction (1) and Final Mock-Up (2) 


3. RESULTS 

In order to analyse the 16 3D future classrooms mock-ups created by the participants, the research team 
looked at two main sources: i) the concepts, images, icons and symbolic pieces of the toolkit used by the 
groups, as well as the different activities zones created inside the classrooms space, and ii) the groups’ oral 
presentations, video recorded. Part of the results that were found are presently reported, specifically: the 
keywords selected by the teachers to describe their vision of what a ‘future classroom’ should be, the type of 
learning activities and corresponding zoning, the psychical elements (furniture and equipment) and the 
environmental aspects (such as light, sound, temperature, etc.). The number of concepts, images, icons and 
symbolic pieces related to each of these dimensions that were used by the teachers in their future classroom 
mock-ups were quantified and the total frequencies are presented in the following graphics. 
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Considering the 3 principles and 15 keywords provided by the research team, it was possible to see that in 
all groups the most often selected principle was Pedagogy (100%). Teachers explained that without serious 
changes in pedagogy, any change in space or any investment in new technologies would be worthless. The 
most selected concepts were Autonomy, present at all the mock-ups (100%) and Collaboration (10/16, 
therefore, 62.5%). Teachers highlight that regular classrooms tend to be organized with a layout that is 
mainly oriented to individualized learning and that it is also necessary to have learning spaces that support 
collaborative activities. Teachers referred that today, and more intensively in the near future, students must 
be stimulated to work autonomously as well as to work in teams and to communication with each other; 
therefore, classrooms should reveal the flexibility to effectively contemplate these different working modes. 



Figure 2. Keywords Frequency Graphic 


The analysis of the spatial organization of the 16 mock-ups results reveals to be congruent with the 
previously described concepts. Teachers organized the classrooms mock-ups around different working areas 
and the most represented area was the one related to collaborative work; the ‘group activities’ zone was 
present in 87,5% of the 3D future classrooms mock-ups. It was also highlighted the benefit of having an area 
for students to work with technologies, mostly referred tablet and interactive/multitouch tables, as well as an 
area for Projection activities, with multiple display technologies that could support the presentation of 
multimedia educational content, by the teacher and by the students. It is also important to notice that in 50% 
of the mock-ups informal learning spaces were put inside the formal learning space that classrooms are by 
convention. The results also shown that, although most often, the layout of classrooms mock-ups was 
oriented by a zoning approach, where the classrooms were divided into smaller areas, 6 of the mock-ups 
(37.5%) represented the future classroom as an open space, where students and teachers could move freely 
and where tables and chairs were mostly removed from the picture and/or replaced by workbenches. 
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Figure 3. Frequency Graphic of the Types of Learning Activities/Zones 


With regards to the physical elements displayed in the future classroom mock-ups, specifically furniture 
and equipment, the results evidenced that teachers report cupboards as one of mostly needed furniture in the 
future classroom. This was explained with the idea that todays’ classrooms already lack places for storage 
(students’ backpacks, coats, etc.) and that, in the future, classrooms must have even more educational tools 
and supplies for students to use, and that these tools (analogical and digital) should have a place to be 
securely kept. Cupboards were present in 93,75% of the mock-ups. Different types of tables, or tables that 
could be configurable into different formats, were also referred, more often tables that could support 
collaborative work between small or large groups of students (62.5%). Furniture that could support students 
working in different body positions, as standing desks or puffs, was also presented as a relevant add-on. 
Teachers explain its benefits by describing the huge amount of time that students pass daily inside the 
classrooms, mostly seating down in the same chair, which most often lacks ergonomics. Indeed, chairs that 
are comfortable and that could be adjustable or personalized were also referred as more suitable for the future 
classrooms. 

The results also evidenced the relevancy that technology and digital devices should have in the future 
classrooms. Small, light and portable devices such as tablets and mobile phones were the most represented 
gadgets (56.25%) but other technologies, not so often seen in classrooms, such as interactive and multitouch 
tables, 3D printers and Augmented and Virtual Reality simulators were also represented inside the future 
classrooms models. 
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Figure 4. Frequency Graphic of the 3D Blocks and Educational Technology Symbols/Words 


Finally, the environmental aspects of the future classrooms mock-ups were also examined, specifically 
luminosity, sound, air quality, temperature, colours, physical comfort, naturalness (elements link to nature) 
and people and equipment security. The number of icons related to each of these dimensions that were used 
by the teachers in their future classroom mock-ups were quantified and the total frequencies are presented in 
the figure 5. 


16 



Figure 5. Frequency Graphic of the Printed Symbols Representing Environmental Properties 

It is possible to see that teachers took in consideration the need for improving the level of comfort of the 
classrooms. 75% of the mock-ups included elements that represented concerns with the physical comfort 
provided to the classrooms users. 68.75% of the mock-ups also revealed the need to improve the luminosity 
of the classrooms, more specially teachers referred the need for more natural daylighting. With the same 
percentage, 68.75%, the future classrooms mock-ups reported the need for a more natural ambiance inside 
the classrooms. 50% of the mock-ups also showed teachers concerns for improvements on the room 
acoustics, more specifically with regards to noise control, considering that communication and team work is 
expected to significantly increase in the classrooms of the future. 
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4. CONCLUSION 

An overall analysis of the mock-ups leads to the conclusions that none of the teachers’ groups built for the 
future a classroom that present the same features of todays’ general classrooms. Though a lot of non-typical 
classroom’ elements were provided in the toolkit, no direct instructions were given for changes to be 
introduced in the classrooms layout, furniture or ambiance. Yet, a significant level of differences could be 
found on teachers’ future classroom mock-ups. One specific group of teachers referred to the mock-ups as 
‘unclassrooms’ archetypes’. The results found in this research project proved that todays’ teachers feel the 
need to shift from teaching in a classroom to teaching in a space fully committed to support new learning 
approaches (NLII, 2004). Teachers’ models of the future classroom showed that teachers claim for a space 
that enables learners to actively manage their learning process and to engage with each other, a space that 
promote autonomy, dialogue and group work. Learning was mostly referred as an active and social process 
and future classrooms must be design with that in mind. Also, the future classrooms should support multiples 
types of learning activities, therefore space must be seen as divisible and furniture must be reconfigurable. 
Digital technologies must be well accommodated and teachers and students comfort must be assured. 

The presence of nature elements inside the classroom, as well as clear concerns with lighting and 
acoustical engineering of classrooms should be addressed. Light can activate students’ attention and improve 
their academic results (Barrett et al., 2015). Also, the quality of auditory perception and control of 
environmental noise clearly improves communication and promotes working efficiency (Hygge & Knez, 
2001; Scanned et al., 2016) and this should be considered as teachers claim for classroom’ designs that 
effectively support collaborative activities inside the classroom space. 

From a practical perspective, these findings are relevant for architects and school boards that aim to 
(redesign) school building and classrooms, yet these are also valuable for teachers that want to improve the 
quality of life of their teaching places. (Small changes can rapidly be made; for example, changing the layout 
of the room, create a visual link between the indoor and nature outdoors or changing the colours of one wall.) 
The findings are also relevant for rethinking teachers’ initial and continuous training as the modernization of 
the classrooms spatial and social environment, as well as its adequacy to the adoption of teaching practices 
that promote active and collaborative learning are still neglected. The further stages of the TEL@FTELab 
aims to address this topic, focusing specifically on teachers’ education programs (on Biology, Informatics, 
Mathematics and Physics) and on how these can increase preservice teacher awareness of the impact detained 
by learning space configuration and elements, analogical and digital, on teachers’ pedagogical approaches 
and daily teaching practices. 
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ABSTRACT 

In most of the pedagogical development experiences, the contents that students generate as a result of the course activities 
are not considered as a primary source of knowledge. Thus, students see their learning disadvantaged, when their 
intellectual outputs are not considered in the design of the learning activities. Today, the Web offers a wide range of 
resources and opportunities for the development of activities in a collaborative environment both to produce or spread the 
contents or to make them available. This article describes the experience of three groups of students from different 
programs, who based in applying a model of interaction among peers indistinctively assume consumer and producer 
knowledge roles, by incorporating authoring tools during their teaching process in a particular subject. Four levels can be 
distinguished for modeling the development of learning activities. Results show that 100% of the students assess 
positively the participation in their role as knowledge producers or consumers, accordingly to the four levels defined 
when using Web tools during their activities. Similarly, they recognize the potential of the instruction among peers 
associated with the use of Web tools as a contribution to their learning development. 


KEYWORDS 

Peer Learning, Education Technologies, Web Tools 


1. INTRODUCTION 

Up to date, classes are structured so that students act as knowledge receptors of what is transmitted by the 
teacher. Following this evidence, some investigations propose a change in this paradigm that consider a 
student as an authoring source of content in the coursing subject, which also allows the development of other 
skills and transverse competences, such as team work (McLoughlin & Lee, 2007; Gray et al., 2010). 

In this context, Internet options to provide students the opportunities to create and publish contents that 
can be potentially used for their peers and their own consumption, is wide and diverse (Bennett, et al., 2012). 
This new way of understanding teaching, through the ICTs incorporation and authoring tools in the 
educational process, opens new possibilities to stimulate meaningful and collaborative learning among peers, 
one present aspect and hence means an alteration in the way of interaction amongst actors in the educational 
process (Rodriguez, Mendoza, 2014). 

Educational institutions recognize the use of Internet promote interactions networks by allowing students 
to develop learnings from contents generated by the own students, and be considered as a primary learning 
source (Staines & Lauchs, 2013). 

Instruction among peer using Web 2.0 authoring tools allows an active interaction beyond the traditional 
classroom. The use of these tools enables the promotion of creativity, and also encourages the interaction for 
achieving team work, both considered transcendental to the modern professional lifestyle. Lasri (2008) 
recognizes the importance of considering the participation of students in designing the learning activities, for 
example, when they interact with their peers inside and outside the classroom to promote their learnings. 

This is the framework that supports the development of this research: “the knowledge or contents 
produced by students are not considered as primary source of knowledge”, mainly because the traditional 
paradigm considers the teacher as the only person empowered to produce knowledge. The intervention 
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shown is based on a model using Web 2.0 authoring tools which by relieving in the students the roles of 
producers and consumers of knowledge enhances the learning process. The model proposed here identifies 
four levels of interaction among peers for the production and consumption of knowledge, which are later 
described. 

This article describes a case study of a group of students that act as knowledge producers, and when using 
learning resources generated by them self or by their peers as knowledge consumers. The aim of the 
experience is focused on promoting, among students from a higher education institution, the use of Web tools 
to produce knowledge. This knowledge is then lately made available to their peers and other group of 
students from different academic programs to be used in learning activities. 


2. PRODUCTION AND CONSUMPTION OF KNOWLEDGE MODEL 


Student’s production of knowledge responds to a simple model. In this model (see Figure 1) the student 
produces contents by using tools from the Web 2.0. 


Contents 


Web 2.0 
Feedback 



FEEDBACK 

- Pedagogical aspects 

- Grammar 

- Format 


Figure 1. Knowledge Production Model 

The content produced is reviewed by the teacher, who provides feedback to the student as often as 
necessary. Thus, as a product from this cycle, the primary content that is available to other students is 
obtained. 

Consumption of knowledge model, was defined based on four interactions levels (see Figure 2). 



Figure 2. Levels of Consumption and Production of Knowledge 


Level 1: Knowledge production level. Students assume the role of primary knowledge producers which is 
going to be used to promote their own learnings and can potentially use by their peers. At this level student 
autonomous working time is optimized, which allows the production of knowledge. 

Level 2: Knowledge consumption level. Students consume the knowledge produced by their classmates. 
The value of the primary contents generated by the students is recognized when the teacher intentionally 
incorporates them in learning activities on a particular subject. The teacher, as a facilitator, generates the 
relevant activities that allow this level of consumption. 

Level 3: Knowledge consumption level. The knowledge consumed is produced by students from other 
programs who attend equivalent courses. For this purpose, the courses involved have a related guidance but 
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their curricula are different and therefore their learning context may have other approaches, even though the 
sequence of disciplinary subjects has common elements. This level must be deliberate by the teachers 
involved, for example through a Virtual Community of Practice generated for this propose. 

Level 4: Knowledge consumption level. In this level the content produced by students is consumed by 
Web users, where massification of knowledge occurs transcending beyond the borders of the classroom. In 
this level a motivational aspect is highlighted, since the student can see that their knowledge production work 
is valued by external users. This is evidenced, for example by the number of visits count, comments posted 
and the amount of “likes” on the contents published on the Web, among other indicators. 


3. METHODOLOGY 

The present work describes an experience that involves different university programs, in which students use 
Web 2.0 tools to generate learning content, which are considered in their curricula and are potentially useful 
to their peers and Web users, as applicable. 

The sample of participant consist in 51 students from all three programs. Pedagogy in Natural Science 
and Biology, Agronomy and Renewable Natural Resources Engineering careers (see Table 1). These three 
programs share a common initial formation subject incorporated into their curricula. In daily practice, there is 
no interaction between students from the different academic programs. 


Table 1. Sample Composition 



Pedagogy in Natural Science 
and Biology 

Agronomy 

Renewable Natural Resourc 
Engineering 

Numbers of Participants 

18 

14 

19 


All participating students from this experience are initially considered as potential actors in their roles as 
both producers and consumers of knowledge according to the levels described in the model. 

For the purpose of this work, available Web tools were classified in the following categories: 
collaborative editing, mental maps, social networks, Web presenters, file sharing. In addition, for this 
experience, its use is complemented by linking an LMS Moodle (Learning Management System) which 
centralizes resources and activities for the student of the course. Criteria used for the selection of the tools 
were: accessibility, usability, gratuity and option for collaborative work in a synchronously and 
asynchronously way among users. Selected tools are shown in Table 2 and its use is specified given the 
context of this research. 


Table 2. Selected Web 2.0 Tools and the Use Given in the Learning Process 


Tool 

Collaborative 

Editing 

Mental Maps 
Diagrams 

Social 

Networks 

Web 

Presenter 

File 

Sharing 

Goear 





X 

Google Docs 

X 





Dropbox 

X 





Slideshare 




X 


Dipity 


X 




Cacoo 


X 




Prezi 




X 


Facebook 



X 



Glogster 

X 
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Based on Lasry’s (2008) “Peer interactions”, the experience for those students attending Organic 
Chemistry subject, consisted in the usage of some of these Web 2.0 tools for specific content authoring which 
became a learning input among students of the same course and students of equivalent courses. The selected 
tools for this work can embed objects on another platform. In this case, the products generated by the 
students were embedded within the LMS Moodle platform. Table 3 shows Web tools used by different 
students as producers and as consumers of knowledge. 


Table 3. Authoring Tools According to Use and Career to Which Students Are Assigned 


Program 

Tools 

Use 

Agronomy 

Google docs, Glogster 
Slideshar, Goear, Facebook 

Produce 

Consume 

Produce and Consume 

Renewable Natural 
Resources Engineering 

Google docs 
Slideshare, Goear 
Facebook 

Produce 

Consume 

Produce and Consume 

Pedagogy in Natural 
Science and Biology 

Dipity, Glogster 

Goear, Cacoo, Slideshare, 
Prezi, Dropbox 

Produce 

Produce and Consume 


Depending on the use of tools and according to what was stated in the description of the proposed model 
of use, a group of students assume the role of knowledge producers, whenever their materials can be used as 
primary source of content by themselves, by students from the same career and by students form other 
programs. The roles of consumers are recognized to other students from the same career or from another 
academic program, which use this material generated by their peers to benefit their own personal learning 
process. In general terms, the student’s work was to select information, organize ideas, prepare presentations, 
and choose a broadcast medium for their peers, both physically and on-line, of a mandatory thematic content 
considered in the indicated subject. 

As already noted, we worked with three programs of the same level, but with different educational 
contexts (Pedagogy in Natural Science and Biology, Agronomy and Renewable Natural Resources 
Engineering). For each program we worked with the Organic Chemistry subject, in a form that would cover 
the production and consumption levels, working in a common thematic content. Production and consumption 
of primary knowledge activities were assigned among the described students groups following the working 
model levels proposed in this work. 

This experience gazed at the development of activities differentiated according to the role of producers 
and consumers of knowledge of students assigned to the different programs, but who share a common initial 
formation subject. The activities in each group and how they are integrated into the learning processes of the 
three programs are detailed below. 

3.1 Pedagogy Students 

For the Pedagogy in Natural Science and Biology program, levels 1, 2 and 4 of the described model were 
worked. Students using different Web 2.0 tools, produced learning products which were used as source of 
knowledge for themselves and by students of this program. Besides, it was found that many of the learning 
products generated, and published on the Web, were consumed by external people outside the institution 
(level4). Table 4 details each of the activities developed. 


Table 4. Activities Developed by Pedagogy Students, Levels 1, 2 And 4 


Activity 

Tool 

Description 

Creating a 

presentation, and Web 
publication 

SlideShare 

Designing a presentation in power point format, with the topic 
“intermolecular attraction forces” which was uploaded to the SlideShare 
platform. 

Timeline creation 

Dipity 

Development of the organic chemistry historical evolution, in timeline 
format. 
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Podcast creation with 
nomenclature rules 

Goear 

Audio files were recorded based in nomenclature rules for aliphatic 
hydrocarbons, which were published in the Goear platform. 

Graphic organizer 
creation 

Cacoo 

Mental maps graphic format were designed in, according to basic concepts 
of organic chemistry. 

Poster creation for 
experimental results 

Glogster 

A collaborative report of experimental results was created in poster format 
according to organic chemistry functional groups. 


3.2 Agronomy Students 

Agronomy students, besides using the learning products generated by Pedagogy students (level 3), produced 
knowledge for themselves and their peers (levels 1 y 2). Table 5 details each of the activities developed. 


Table 5. Activities Developed by Agronomy Students, Levels 1, 2 and 3 


Activity 

Tool 

Description 

Solving organic nomenclature 
exercises 

Google 

Docs 

This group uses pedagogy student’s podcast to solve the exercises in a 
shared Google Docs spreadsheet. They work identifying and 
denominating chemical compounds applying nomenclature rules for 
organic compounds. 

Presenting experimental work 
results 

Glogster 

After an experimental work in the organic compounds physical 
properties laboratory, students published their results in a poster 
crated in a collaborative way, in groups of three students. Each poster 
was published in Moodle to be used by all students. 

Investigating and generating 
content to describe properties and 
characteristics of organic 
compounds 

Slideshare 

Students organized in pairs. They worked on an assigned topic to 
investigate and create a presentation that should be published on the 
Slideshare platform. 

Learning biosafety rules for 
laboratory 

Goear 

A document with the laboratory biosafety rules was given to the 
students. They selected one of those rules and created a podcast later 
uploaded to Goear and published in Moodle. 


3.3 Engineering Students 

Natural Resources Engineering students used the learning products created by Pedagogy students, and also 
prepared learning products for personal consumption and for their peers. Table 6 resumes the activities 
developed by the Engineering students and the Web tools that were used, which stablished the production 
and consumption of knowledge at levels 1, 2 and 3. 


Table 6. Activities Developed by Natural Resources Students, Levels 1, 2 and 3 


Activity 

Tool 

Description 

Fill multimedia template to 
recognize laboratory tools 

Google Drive 

Given a template prepared by the teacher and shared Google Slides, students 
worked on the development of this to describe laboratory instruments, 
depending on use and/or application. 

Study and resolution of 
naming rules exercises of 
nomenclature for organic 
compounds denomination and 
formulation 

Goear 

Audio files created by Pedagogy students were embedded in Moodle so that 
Engineering students could listen repeatedly the naming rules, and lately 
apply these rules in resolving the organic compounds denomination and 
formulation exercises guides. 

Organic compounds 
characterization 

Google Drive 

In a Google Docs shared document, students incorporated and/or collected 
from the web new bibliographical contributions that allow them to 
characterize chemical compounds according to observations of the 
experimental work. 
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3.4 Evaluation 

To perform an analysis of the model and their levels, a survey was made in Likert scale format (register 1 to 
5), in which 5 is “strongly agree” and 1 is “strongly disagree”. This instrument was randomly applied to 
students of the different participating programs. The questions were designed to show results from the four 
levels defined in the model. Table 7 shows the questions of this survey, classified depending on the definition 
in the proposed levels for the mentioned model. 

Table 7. Survey to Assess Use and Satisfaction, According to Levels 


1= Strongly disagree 2= Disagree 3= Undecided 4= Agree 5= Strongly agree 


Level 

Questions 

1 

1. The production of contents that were used by peers motivated me to create a good quality learning 
product. 

1 

2. The Web tools used to crate contents are important to develop my own learnings. 

1 

3. The activities implemented with these tools are motivating to develop learnings procedures. 

1 

4. The strategy of creating material for my consumption and for my peers’, favors the development of 
learning. 

2 

5. The material produced by my peers contributed positively the process of the programmed activities in the 
subject. 

2 

6. Using contents created by my peers inspires distmst. 

2 

7. The production of learning content that will be used by my classmate favors my own learning in this 
discipline. 

3 

8. Using contents crated by pedagogy science students was useful to develop my own learnings. 

3 

9. The material developed by Pedagogy students is of a suitable quality for my subject requirements. 

3 

10.1 appreciate positively using materials created by Pedagogy student. 

3 

11. Web 2.0 tools that Pedagogy students use are suitable to develop my learnings. 

4 

12. Learning contents that are visible on the Web are an important stimulus for developing my future 
professional skills. 

4 

13. Knowing that my content produced with Web 2.0 tools will be posted on the web motivates me to 
develop good quality material. 


This tool was individually applied to each student through Google Form, using LMS Moodle platform to 
distribute it. 

The data collected was systematize by using a simple mathematical model for its analysis and further 
discussion. 

4. RESULTS 

Tables 8 and 9 show the results of the survey applied, with the identification of the assessment of the 
experience in relation to the levels defined for their implementation. 
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Table 8. Results of the Survey Question by Question. Data Mean According to Levels 



Level 1 

Level 2 

Level 3 

Level 4 

Question» 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

Pedagogy 


4,7 


4,7 

4,1 

2,8 

4,7 

- 

- 

- 

- 


4,7 

Agronomy 



4,0 

4,7 

4,3 

2,9 

- 

4,0 

3,9 

3,9 

4,1 

- 

- 

Engineering 

- 


- 

- 

- 

- 

- 

3,9 

4,0 


3,9 

- 

- 

Mean 




4,7 


2,8 

4,7 

3,9 

4,0 

4,1 

4,1 


4,7 

i 4,5 ! 

3,9 i 

4,0 

4,5 i 


Table 9. Results of the Survey. Overall Mean for Each Program According to Levels 



Level 1 

Level 2 

Level 3 

Level 4 

Pedagogy in Natural Sciences 

4.6 

4.4 

- 

4.7 

Agronomy 

4.4 

4.3 

4.0 

- 

Renewable Natural Resources Engineering 

4.3 

- 

4.0 

- 

MEAN 

4.4 

4.4 

4.0 

4.7 


It is noteworthy that for all levels activities were assessed positively (rank “agree” and “strongly agree”). 

Level 1 (the student produces knowledge for self-use) was developed in all programs. Students assessed it 
very positively. 94% of the answers obtained values positively (score 4 or 5) being user and consumer of 
knowledge created by themselves. Besides, this shows that students assess positively the incorporation of 
Web tools to produce good quality content that promotes their own learning and peers’. 

Like in the previous level, in level 2 (students consumed knowledge produced by peers from same course) 
the assessment from the students is positive. From this we can validate the model of knowledge production 
(Figure 1.), whenever students recognize the value of contents created by their peers. This content is made 
available to users throughout a teacher's mediation, who intent their use to promote the learning of others. 

In level 3 (students consumed knowledge produced by peers from other programs). Agronomy and 
Engineering programs consumed knowledge created by Pedagogy students. Students from these programs 
valued positively the contents crated by their peers from Pedagogy. The teacher's intervention as mediator of 
the contents to students is outstanding, and the results enable to validate the model described for the purposes 
of this study. 

In level 4 (contents created by students is consumed by external users from the Web) the instrument was 
applied only to pedagogy students. This level was positively valued, which shows the satisfaction of students, 
who were producers of knowledge, when observing that the contents of their authorship was consumed and 
valued by external users. Results show that content massification created by students, through the Web, 
reinforces their self-esteem and recognizes its value as knowledge producer. 

Additionally, based on the obtained classifications from different courses it was possible to stablish that 
students achieved better grades compared to other semesters without this intervention. 

Furthermore, development of transversal skills like team-work, autonomous learning and knowledge 
management is strengthened. This can be achieved with the use of technology due that most tools offer 
options that point to collaboration. For example, collaborative creation of conceptual maps with Cacoo, the 
creation of collaborative documents with Google Drive, or the creation of collaborative posters with 
Glogster, among other tools. 

On the other hand, at general level, class dynamics and the personal motivation to face and assume their 
training successfully are strengthened. It was possible to value the quality of the knowledge created by the 
students based on comments, number of visits, amount of “likes”, among other indicators. This strategy 
enhances student motivation, because students can perceive that their contributions are valued by others. 
Besides, it motivates them to generate good quality products, knowing that they will be exposed to trial by 
others on the Internet. 
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5. CONCLUSIONS 

Throughout this study, we can confirm that the education among peers through the information and 
communication technologies is a useful tool that benefits and supports learning, and above all, motivates the 
student to produce good quality products. This is because there are dozens of tools for multiple use, available 
to teachers and students that are socially recognized and that incorporate social interactions elements like: 
collaboration, resource creation, share work, resource and experience feedback, and direct communication 
between people, among others. Therefore, it is possible today to take advantage of all these possibilities and 
highlight the role of the student as a producer of knowledge that will be used by their peers. Thus, training 
among peers using Web 2.0 authoring tools, allows an active interaction beyond the traditional classroom 
that helps to develop good quality learnings in different subjects such as chemical sciences and in areas like 
pedagogy and agronomy. 

According to the results and considering the students’ participation depending on different production and 
knowledge levels described in this article, it can be set that students from the three programs assess positively 
the proposed intervention according to the model of producers and consumers of knowledge in their different 
levels. Furthermore, it was possible to prove participation of students at all levels of producers and 
consumers of knowledge described. In this sense, the benefit on students is displayed, assuming both roles, 
producer and consumer. The role of producer allows them to incorporate, in the development of activities of 
a particular subject, the use of Web tools to strengthen their learning and to develop autonomy. On the other 
side, in their role of consumers, students showed satisfaction in the use of these strategies, assessing 
positively the knowledge produced by their peers. 

The interaction among students, allowed the discussion of the issues discussed and the recognition of the 
contributions that each one of them could present independently to enhance their learnings. Moreover, this 
interaction allowed the detection of the best rated contents by the students themselves, identifying the most 
visited contents and those who were consulted more times. Thereby, students were able to recognize good 
quality job among their peers. 
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ABSTRACT 

Perspectives on the use of technology in teaching and learning have been increasing polarised, with positions entrenched 
around the efficacy of using technology in lectures, the distractions assumed to arise from social media and the 
temporality and ephemerality of knowledge located outside the academy. This paper presents the preliminary collective 
findings from several analytical projects arising from an innovative consultation project at the London School of 
Economics called LSE2020. This project was a central component of how we designed and delivered the strategic 
implementation of pedagogical change at the School. It sought to identify barriers and opportunities that can emerge from 
the integration of technology at a curricular and delivery level. The primary finding of the study is that students used and 
valued the effectiveness and benefit of the technologies that were provided to them by the institution and technology they 
chose to use themselves in different ways. The technology provided by the institution such as the Virtual Learning 
Environment and lecture recording facilitated actions aligned with the necessity to succeed, whilst their own technologies 
were part of wider approach to understanding and coping with the intersecting pressures of personal, professional and 
educational lived experiences. 


KEYWORDS 

Technology Enhanced Learning, Pedagogical Change, Social Media, Higher Education 


1. INTRODUCTION 

There is a growing body of research and practice informed literature that argues against the efficacy of 
technology to support effective student learning and academic practice (see e.g. Rockmore, 2014, 
Rosenblum, 2017, Holstead, 2015, Ravizza et al., 2017, Patterson and Patterson, 2017, Mueller and 
Oppenheimer, 2014). Many of these studies start with the premise that student use of technology and social 
media (in the narrow confines of face to face teaching) distracts from and diminishes their learning (Taneja et 
al., 2015) or represents an inferior way to engage in learning activities in the classroom (Mueller and 
Oppenheimer, 2014). With topics ranging from the positive benefits gained from banning laptops in a lecture 
theatre through to the distractions inherent in the apparent overt engagement with social media by students 
during lectures, these articles have set out battle lines in what could be described as a fake war between 
protectors and challengers, defenders of the faith versus the barbarians at the gate (Anonymous, 2016, 
Rockmore, 2014, Rosenblum, 2017). As Aran Levasseur (2011) argues in the article ‘Teaching without 
Technology’ in the wider context of school education: 

The conflict between computers and schools is really a conflict between educational paradigms. The 
traditional and dominant paradigm is rooted in the book and the pedagogy is one of transmission. Teachers, 
who have presumably read more books than their students and listened to more scholarly lectures, transmit 
what they’ve learned to their students in a similar fashion. The students who do best within this system are 
those who can capture the transmission — as unfiltered as possible — and mirror back to the teacher what 
they have delineated. Within this model, digital technology can provide improvements, but they are cosmetic. 
(Levasseur, 2011). 
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The rebellion against the use of technology described in many of these articles is underpinned by an 
argument that asserts that it is the academic that knows what is best to encourage and enhance student 
learning in the classroom. This is framed within an argument that there are behaviours that the academic 
adjudges to be educationally relevant and the these behaviours are undermined by the distraction of news, 
images, selfies and shiny things that are assumed to proliferate within the students use of technology such as 
laptops, phones and social media (Aagaard. 2015). The academic is argued to have a privileged role; part 
distributor of established knowledge, part assessor of scope and scale of learning and part martyr for the 
cause of student learning defining what is a distraction and how technology is assumed to be used (Roberts 
and Rees, 2014). Some writers argue that the academic becomes the ‘police officer’ for preventing distracting 
technology use and thereby enhancing student learning (Gupta and Irwin, 2016, Selwyn, 2016, Wright, 
2016). There are several examples where the academic has advocated for and delivered the shutting down the 
Wireless network, punishing students for laptop use or the confiscation of devices from students (Baker et al., 
2012, Hanson et al., 2010, Berschback, 2010). Almost of all the examples in the literature are predicated on 
there being a dominant form of teaching (didactic lecturers) that requires the student to learn through 
consumption of knowledge and listening. The teacher holds all the knowledge and the only way students can 
obtain it to listen and note-take. A good example comes from an anonymous academic publishing in The 
Guardian in 2016, who notes: 

When did it become acceptable to use your phone throughout a lecture, let alone an entire conference? No 
matter how good you think you are at multitasking, you will not be truly focusing your attention on the 
speaker, who has no doubt spent hours preparing for this moment. (Anonymous, 2016) 

There are several critical assumptions inherent in this paragraph, around what the student is using the 
mobile device for and the relative politeness arising from its use in the lecture, the capacity of individual 
learners to multi-task in a digital world and the importance of the words and actions of the speaker to the 
learner (not to mention the quality of the teaching of speaker involved in terms of how effective they are at 
communicating and sharing knowledge and understanding). This argument positions technology as the ‘bad’ 
force and the users of it as people who are rude, insensitive or ungracious for the efforts of the speaker 
concerned. Holstead (2015) makes similar assumptions about the disruption to learning that arises from the 
use of technology to multi-task. Drawing on a hypothesis from a pre-digital age, the author asserts that 
laptops encourage students to try and transcribe lectures in their entirety whilst pen and paper supports more 
holistic, structured note-taking which the author concludes enhances the learning outcomes for her students: 

I knew that eliminating laptops in my classroom would reduce distractions. Research has shown that 
when students use their laptops to "multitask" during class, they don’t retain as much of the lecture. But I 
also had a theory, based on my college experience from the dark ages—the 70s, aka, before PowerPoint— 
that students would process lectures more effectively if they took notes on paper. When students took notes 
on laptops they barely looked up from their computers, so intent were they on transcribing every word I said. 
Back in my day, if a professor’s lectures were reasonably well organized, I could take notes in outline format. 
I had to listen for the key points and sub-points. (Holstead, 2015) 

There are common threads running through these articles in terms of the attitudinal and behavioural 
aspects of the teachers and students under study (although it is interesting to note that only a small number of 
these studies or articles talk about or to students directly, referring to them generally in the abstract). Firstly, 
there is the argument that either explicitly or tacitly the modern learner engages in behaviours that are 
different to the ones they exhibit in their professional practices or used during their own studies (as exhibited 
by Holstead). Secondly, there is the assertion that social media is used by students to waste time, as a 
distraction from the learning being presented from the podium or as a way of avoiding learning altogether 
(what Taneja et al (2015) labels as cyber slacking). Egan (2016) makes this case explicitly arguing that social 
media only serves to engage students in superficial, identity driven social interaction: 

We should be teaching students computer programming so that they can use these machines in ways 
limited only by their imaginations and effort. We should be discouraging them from wasting their days 
fretting over the trivial details of who thinks what about whom in their group, updating their "statuses", and 
sharing the pictures that they think puts them in the best possible light (Egan, 2016). 

Facebook is cited frequently as the platform that distracts students the most from the learning in lectures 
and classes (Rosen et al., 2013, Bugeja, 2007, Gupta and Irwin, 2016). Sprensen (2014) illustrates this point 
by assuming that all Facebook use in his classes (25-50% of students from his research) competes with the 
academic for the attention of students: 
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The fact is that a large proportion of students are not mentally present during class. This then leaves the 
rest of the student group permanently distracted by those “leaving class” via social media: the tantalising 
Facebook logo popping up on some students’ screens will inevitable fuel everyone’s fear of missing out. The 
teacher is then left to the merciless competition with the social media for the students’ attention - a 
competition even the most gifted teacher will eventually lose. (Sprensen, 2014) 

In this introduction, I have presented some of the counter-arguments to the efficacy of technology and 
social media in higher education as a demonstration of a seemingly growing trend in the literature and the 
popular and educational press. Clearly, there is a strong tradition of literature that explores and interrogates 
how technology and social media can enhance the learning for students (see e.g. Nykvist and Lee, 2013, 
Caplan et al., 2014, Greenhow and Lewin, 2016, Berger and Wild, 2016, Ada et al., 2017). What is 
interesting is that many of these studies and the hundreds similar are not offered as counter cases to the more 
personal assertions of the negative impacts of technology, instead they are presented as the accepted lore of 
modern teaching for which the anecdotal and sometimes scientific evidence they present attempts to counter 
(Barry et al., 2015, Prestridge, 2012). 


2. LSE2020 AND THE STUDENT USE OF TECHNOLOGY IN LEARNING 

At the London School of Economics and Political Science, in the United Kingdom (LSE), we have been 
engaged in a strategic process of pedagogical change designed in response to several critical drivers for 
enhancing teaching and learning in the modern age. Using a design thinking approach to inform the strategic 
and operational approaches we undertook (Meinel and Leifer, 2010), the problems of student satisfaction, 
teaching quality, the balance between research and teaching and increasing global competition were 
visualised the university as a series of overlapping spaces, representing where students, academic staff, 
professional services teams and society reside and intersect in teaching and learning. Change was approached 
not as top-down or bottom-up initiative driven by managerial objectives or the enthusiasm of the engaged, 
but as a design process of identification, experimentation and collaboration, with change as the wicked 
problem at the centre. Critical to this process was the necessity of finding commonality within those spaces, 
with technology a contradictory and challenged piece of the puzzle. 

Essentially a face-to-face experience, teaching at the LSE utilises technology in inconsistent ways, 
ranging from simple replication of analogue practices such as media broadcasting and dissemination of 
materials such as slides or notes through to highly engaged and innovative student-led creation and 
production of knowledge projects. Students actively engage with the systems and platforms provided for 
them by the University, with successful institutional and student engagement with the Virtual Learning 
Environment, originality reporting and lecture recording over the last ten years. Student feedback including 
the National Student Survey consistently values the institutional technology and demands the increased use 
of it by academics, locating those technologies at the very heart of their success in a complex and competitive 
higher education experience. Presented with challenging metrics of student satisfaction and teaching quality, 
the School identified technology as a critical lens to understand and deliver essential and necessary 
pedagogical change. We developed a strategy called Learning, Teaching and Technology Futures which set 
out to build an engaged and critical learning and teaching community at the LSE and reconfigure the student 
experience at the School. Critically, it was not a e-learning strategy or a technology enhanced learning 
strategy. The strategy was a framework to understand the interactions between people and technology, the 
resistances and affordances that emerge from the use of technology and technology informed practices in 
education. It recognised the complexities of the overlapping spaces within the School, with implementation 
models such as compliance, coercion, opting-out or quality assurance providing little or no scope for a 
successful transformation. 

The first stage of implementing this strategy was to identify the environmental factors that slowed down 
or potentially resisted the scaling up and sustainability of pedagogical change through technology. Myths 
build up within institutions about the efficacy of technological interventions, designed in part, to resist 
engaging in processes that challenge current practice or require significant re-thinking of the ways in which 
teaching and learning are enacted (Bryant et al., 2014). These myths create walled gardens of practice, where 
the calls for change are challenged by anecdotal assertions, msted-on custom and practices, institutional 
inertia and sometimes outright resistance. Taking an almost adversarial perspective to the use of technology 


69 



ISBN: 978-989-8533-71-5 © 2017 


by both students and staff, interventions by the institution to effect change can become polarised debates rent 
with assumptions about how technology is used and for what purposes (a case seen to great effect in some of 
the articles discussed in the introduction). Instead of engaging in the more traditional consultation processes 
(listening exercises, working groups, committees etc.) to address those concerns and to design the strategy to 
respond to them, we took a different approach. We undertook a civic engagement informed approach that 
draws on the principles of crowd learning, digital citizenship and social media practice to better understand 
the spaces in which people engaged in learning. We planned a series of innovative interventions using a 
variety of different methodologies, such as hacks, crowdsourcing, conversations, debates, provocations and 
media-making to involve and give ownership to staff, students and the LSE community in the programme of 
change. We decided to start this conversation with the students. Both in the literature and from the anecdotal 
experiences we had heard coming from our students, the idea that education as something that needed to be 
‘done’ to students, sometimes unwillingly was an assumption we wanted to challenge. Social media was a 
distraction, laptops diminished their capacity for successful learning, the Internet was a site for cheating and 
essay mills, plagiarism detection could be ‘gamed’ if it was made available, lecture recordings decreased 
attendance at lectures. It was critical to expose the assumptions behind these assertions and understand how 
students used technology for the learning and how their use of technology bled between their contexts of life, 
education and work. 

LSE2020 is an innovative programme designed to be the catalyst for these conversations. The intention of 
LSE2020 is to provide students the opportunity to engage in conversations, discussions and debates with each 
other and with the School. Through recording and distributing these conversations on social media with 
people outside the institution, we added a sense of identity, sharing and critical thinking to the conversations. 
Filming conversations, group discussions and more formal interviews with nearly two hundred students over 
two years, LSE2020 focused on how students at the School use technology and social media for their 
learning, their career and across the ways they choose to live their lives. What emerged were authentic stories 
of how LSE students study, how they engage with other students both inside and outside the School and what 
it means to be a modern student at the LSE. LSE2020 identified how students engaged in collaborative 
practices such as Google Docs being used for collective lecture note-taking, professional and personal 
identity (the use of Facebook for group work and peer learning) and the de-location of study from the 
physical campus into social media apps like Snapchat, Whatsapp and Instagram: 

So, I mean, we’re able to connect with each other on another level than if like I would have during 
undergrad and I think a huge part of that is through social media platforms like Facebook where you’re able 
to, you know, if we have an event coming up in our core of work, like she and I will both know about it, we 
don’t really have to chat with each other, we’ll see it on Facebook which is incredibly helpful (Student 
interview from LSE2020 stage 2). 


3. METHODOLOGY 

Starting with 182 three-minute video conversations conducted between 2016 and 2017 and adding a survey 
that asked similar questions of a further 250 students in 2017, LSE2020 generated over 150000 words of data 
centred on how students use technology to engage in and support their learning for LSE courses and 
programmes and more widely how it intersected their personal, professional and educational lives. The way 
these conversations evolved was critical, as they were not based on a question/response model, interviewer 
and interviewee taking specific roles active and passive roles. Instead, the use of students and recently 
graduated students (working as interns) as conversants within the research design and the process itself of 
engaging in filmed but informal conversations facilitated a sense of openness and resonance in the responses. 
The results presented here are a snapshot of several intersecting analytical projects looking at learning, 
wellbeing, digital capability, normalisation and literacies. These studies are using several different 
methodological approaches including grounded theory, content analysis and thematic analysis to extract the 
insights from the interviews and conversations. This preliminary study collates the findings from these 
on-going analytical projects, along with the more strategic and operational interpretations that came from the 
project. Ultimately it is critical to note that LSE2020 is not a research study. It is designed to inform 
pedagogical change at the School and engage the students as active participants in that change. These 
students provided information and insights to the project to better inform their own and future colleagues 
educational experiences. It is through that lens that the data analysis used in this paper drew its conclusions. 
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4. FINDINGS 

Two main themes emerged from the data, centred on the technology the institution enforced or compelled 
them to use and the technology they chose to apply to their own learning. The first, emerging primarily from 
the first stage of the project started in 2016, looked at the technology and practices the institution provided 
for students and how they engaged with the learning activities initiated by that technology. The second theme 
emerged from the second stage in 2017. Defining technology and social media use in a more holistic sense, 
entangled in the student’s wider contexts of work and life, LSE2020 explored the technology and social 
media practices the students brought to their learning and how they used those technologies to undertake 
learning activities. 

4.1 Understanding our Technology and what we want them to do with it 

Most institutions provide students with a variety of higher education specific technologies to complete 
specific tasks required for verification, dissemination, assessment and participation. These technologies are 
often specific to the education sector, but draw on more established universal principles and practices for 
using technology (communications and file sharing for example). The first LSE2020 interviews focused 
almost exclusively on these technologies, although not by design. When asked questions about what 
technologies they used for learning, students defaulted to discussing the technologies the institution had 
given them or that their academic used in their teaching and assessment. We found that students only were 
willing to describe the platforms that were directly related to their teaching and learning experience. To that 
end, students did not know what and how technology could be used to enhance teaching and learning both 
now and in the future. In simple terms, students used the institutional systems to support achievement, obtain 
the grade they needed and to effectively complete the requirements of the course (Liote and Axe, 2016). 
Where they were not exposed to different educational technologies over and above what their teachers used, 
they had no imperative or capacity to find out more about that technology. This was especially prescient as 
75% of the students observed that their teachers used no technology at all or just PowerPoint, although every 
course at the School has a Moodle site and 65% of undergraduate lectures are recorded and made available to 
students: 

I don’t think we understand what we are missing out on in any way because [...] we don’t know what 
technology is available and how it is changing (Student interview from LSE2020 stage 1). 

You only know what’s possible if you’ve seen it in the first place. I don’t really know what alternatives 
there are to PowerPoint (Student interview from LSE2020 stage 1). 

Where the students were exposed to technology in their teaching (aside from PowerPoint), they suggested 
a vision for teaching that engaged in online and blended delivery, closer and more engaged social media 
contact between students and in smaller numbers, between staff and students and ways to make their lecturers 
more interactive. None of these responses generally challenged the dominant paradigm of teaching and 
learning, but described pathways through it. Despite raising concerns about the look and feel of Moodle (the 
School’s Virtual Learning Environment), the unreliability of some systems and the quality of lecture 
recordings, they did not see their role as one that effected change in those systems. They needed to use this 
provided technology in ways that were necessary, get what they needed and complete the requirements of the 
course. This perspective was reinforced by a student interviewed in stage 2 who noted: 

Moodle. I love because it gives me the opportunity to enrol in any class outside of my programme and 
look at the curriculum and the readings and it’s a really great platform for me to get access to course 
material. I love that they include the reading lists, everything is in one place, so I think it’s easy to use. Um, 
and it has the essential components needed to be, to do well in class but same, yeah like I know Moodle has 
like more functions to chat, discuss but I’ve never used the functions just like to see the curriculum material, 
like downloading material. Basically, for that function only for me (Student interview from LSE2020 stage 
2 ). 

4.2 Understanding their Technology and how they use it 

Insights about the use of technology in learning were not prevalent in the first stage of interviews. The notion 
that students did not know what they did not know necessitated some significant rethinking about the 
process. If LSE2020 was to be a critical conversational pathway to facilitating change and disengaging from 
the them and us antagonism described in the earlier section, the discussion had to allow the emergence of 
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insights into practices and devices that perhaps the students did not think were the exclusive purview of 
education or in fact perhaps did not consider to even be technology. One of the most telling demonstrations 
of this was in Stage 1, where one undergraduate was asked what technology they used for learning and listed 
Moodle and the reading list system. When pressed by the interviewer the student that they used no other 
forms of technology. In front of the student in the video was their laptop, tablet and mobile device, all active. 
The conversations in stage 2 turned from what technology do you use to how do you use technology and 
social media. These question approaches opened up streams of conversation around identity, bullying culture, 
communications, collaborative work, management of time and most critically how learning, life and work 
intersect 

The technologies and practices students engaged with outside of their formal learning activity (lectures, 
classes, readings, assessments etc.) supported metacognition, network development and sociality. These 
technologies were out of the direct control of the institution and rarely embedded in curriculum and 
assessment. These technologies were owned and managed by the student, were multi-functional in that they 
were used for both personal and educational activities. What we found to an almost overwhelming degree is 
that student use technology to communicate, almost to the exclusive detriment of other forms of 
communications. Over 95% of the students we talked to used apps, mobile devices, Moodle, Facebook or 
other social media to plan, meet up, collaborate, do assignments, share notes, verify and validate information, 
collect data and a wide range of other learning activities. Ranging from personal devices such as smartphones 
and laptops through to social media platforms that support interaction, engagement and collaboration such as 
Whatsapp, Facebook, Snapchat and Google Docs, these technologies were not exclusively deployed for 
education. They were part of the way students lived, with devices and applications used for specific and 
identifiable purposes, but not just for study or learning, or engaging with other students: 

I will reach out to social media and ask my friends, okay what do you think about this? Do you agree with 
this? (Student interview from LSE2020 stage 2) 

If I meet someone [...] I’ll never ask for their number I’ll always ask for their Facebook. (Student 
interview from LSE2020 stage 2) 

You’ve got like a Google Doc, everybody can comment on what you write during the lecture, someone 
types what the lecturer says and other people comment on it. (Student interview from LSE2020 stage 2) 

Whilst these short excepts from the interviews are descriptive in nature, there were part of a much wider 
narrative contained within many interviews. One of the students interviews above noted how they used social 
media to validate and verify difficult concept with their friends. Placed back in the context of the wider 
response, the student told a much richer story of how they used technology in their everyday lives, the 
emotions, anxieties and possibilities it created and the capacities it provided to get things done: 

For my studies 1 use my smartphone. For the majority of it it’s my laptop. I look at readings on my laptop. 
I take notes on my laptop. Sometimes side by side I’ll have the readings, the pages I’m taking notes on 
concurrently so I can switch back and forth very easily. If I want supplemental information, I can very easily 
Google up certain things I might have questions about or articles 1 might immediately relate to any 
theoretical concepts that I am studying or practical studies that I’m looking at the supporter, degrade it. I 
also use Facebook when I see a particularly interesting concept that either makes me mad, is quite 
controversial or I really agree with or something that I'm trying to puzzle out. So I will reach out to social 
media and ask my friends, okay what do you think about this? Do you agree with this? Where do you think 
this might be wrong or where do you think it’s strengths are or how controversial the statements are, how 
they are wrong in all the wrong ways (Student interview from LSE2020 stage 2). 

Across this response, which was mirrored by those of many other students, the complexities of life, work 
and study and how they are shaped and conducted through and with technology is apparent. Challenges to 
authenticity of knowledge, the primacy of the voice and opinion of the academic, the criticality of the 
experience of ‘being there’ at the lecture and the importance of the network are all present in a single 
180-word response. Similar assertions were held by a significant majority of respondents who engaged in 
varying degrees of personal and collective reflective criticality about the efficacy, ethics or societal impacts 
of using technology or social media. What was most prescient about their insights was that much of this 
happened despite the deficit of technology use by teachers or the sometimes-active lobbying against it. A 
good example was where students collectively organised through Facebook to take and share lecture notes on 
Google Docs. These were updated and honed by students, tested against the lecture recordings and the link 
shared through back through Facebook. Whilst it might seem that these students were surfing Facebook or 
residing within the Google ecosystem instead of ‘studying’, they were in fact engaging in learning activities 
(the effectiveness of which was clearly not measured by this study). This engagement was collective, did not 
actively involve the teacher to validate or gatekeep the ways in which knowledge was recorded and 
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diversions and distractions minimised and was shared widely with the community of learners, even when the 
technology that the institution gave to them was not working, with one student noting: 

Um, well the lecture recording has been very useful lately but it’s not been working perfectly. (I used) not 
a software but maybe a social media where, a Facebook for LSE where you can turn files that would be like 
helpful to group, to share information. We use that now as a Facebook group for the whole generation of my 
classmates and every time that we want to share a book we have to put the link and make the link bespoke so 
in terms of sharing information orfdes, it could be like easier (Student interview from LSE2020 stage 2). 


5. CONCLUSION 

The strategic pedagogical change that started the need for these types of conversations is by no way complete 
at the LSE. We are only at the very early stages of interrogating how the student-led use of technology and 
how they engage in the technology provided by the institution can make learning better. We have continued 
to take a step back and interrogate what it meant to do an LSE education in the post-digital age. We set out a 
vision for our future that we are still delivering today. The complex, multi-disciplinary, research intensive 
environment located within disciplines and work environments are technologically innovative and demand a 
high level of literacy. Equally, they demand an agile engagement with technology and social media that can 
create tensions, debates, competition for resources. But these tensions can force change to be informed by 
people, by research and by commitment. LSE2020 has begun to identify the information and insights about 
both side of this ‘debate’. The outputs of LSE2020 are revealing the tensions experienced by students and 
staff in ways that make the assertion of unsubstantiated assumptions difficult or redundant. The students 
involved in LSE2020 were not concerned what their academics thought they were doing with their 
technology and clearly wanted to demonstrate what they were doing with ours. They just got on and did what 
they needed to do in order to learn and to succeed. Study was described in sometimes stressful, complex and 
difficult terms by the students. Workloads, time management, efficiency, expectations and anxiety were all 
part of the reason why students sought out technology. Technology was sometimes the only thing that they 
believed kept their head above water in an educational environment demanding not just learning and 
participation, but economic stress, separation from home and family and living and surviving in a city like 
London. 

The way technology is used to collaborate, share, critique, engage shapes the way we communicate. To 
ignore technology and social media and its transformative community of practices would be a dangerous 
ignorance for higher education institutions, mainly because despite all the talk of kill switches and pen and 
paper, technology use by students (and staff will always happen). That doesn’t mean we have to all 
communicate through twitter in 140 characters, nor does it mean that crowdsourcing and Yelp 
recommendations will replace academic knowledge as the purest form of thought. But it is in those very 
defences against using technology that one of the most fundamental tensions in higher education lies; you are 
either with us or against us. It is a polarised debate, with no middle ground and a series of entrenched 
positions backed with rigid institutional structures and policies and with all the risk dumped heavily on the 
shoulders of students. How do students respond to this? Through LSE2020 they told us to use our technology 
better. And then they can be left on our own to study and prepare and learn. What LSE2020 suggested was 
that it did not matter what devices were in their hands, they will be there and they will be used not just for 
study, but work and for life. Perhaps the most important insight for the institution is knowing that what they 
are using and how they choose to use technology and social media empowers both them and us to make 
better learning experiences. 
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ABSTRACT 

For a long time the concept of addiction has been considered within the medical context of consuming certain materials 
excessively. Lately, it has been used to describe by psychologists some behavioral habits that are outside the normal life 
of the general population, such as gambling which affect a particular group of individuals. After the Information 
Revolution, burgeoning developments in digital technologies and corresponding changes in behavioral patterns of users 
have brought new debates pointing the existence of a new form of addiction which has not been observed before. This 
form of addiction has generally been called "technology addiction". Considering that social media addiction is probably 
the most recent kind of technology addiction, the present study was designed based on the six components suggested by 
Griffiths (2013). Toward the purposes of the study, the "Bergen Facebook Addiction Scale" was adapted to social media 
addiction and translated into Turkish. After the validation process, it was administered to a total of 700 students; of them 
397 were high school students and 303 were university students. The data collection instrument included 18 five-point 
Likert-type items in six categories, along with five structured items regarding demographics of respondents. The 
Cronbach’s alpha reliability coefficient for the scale was .90. The data were analyzed through both descriptive and 
inferential statistical techniques. In addition to the findings of the present study, similar research on social media 
addiction in various countries were examined for comparisons. The results of the study showed that both university 
students and high school students have a moderate level of addiction to social media. Being a university or high school 
student does not make any difference on the level of social media addiction However, significant differences were found 
regarding gender, duration of use, department at the university, and type of high school. Finally, the results of the study 
show certain similarities and a few differences with the results of the studies conducted in other countries. Implications of 
the results are discussed and recommendations are made both for future research and improvement of practices. 


KEYWORDS 

Social Media, Addiction, Social Media Addiction, Social Networks, Internet, Technology Addiction 


1. INTRODUCTION 

Traditionally, the concept of addiction is based on a medical model, and it expresses extreme physical or 
psychological desire and commitment to a physical object. However, researchers argue that addiction should 
be addressed to cover a broader behavioral domain (Shaffer, 1996, p.462). On this manner, Griffiths (2013) 
proposed the concept of technology addiction. He has defined technology addiction as the actions which are 
non-chemical but behavioral, and involve extreme human-machine interaction. Also, technology addiction is 
defined as a passive behavior like watching TV, playing a video game, and chatting online. 

Internet addiction is a special type of technology addiction. The term “Internet addiction”, for the first 
time, was defined by Ivan Goldberg in 1995 as a psychological illness (cited in Kim, 2009). Different notions 
were used for abusing Internet. Young (1998) named it as “internet addiction”, Morahan-Martin and 
Schumacher (2000) used the term “pathological internet use”, Davis, Flett and Besser (2002) labelled it as 
“problematic internet use” (cited in Caplan, 2002, p.554). 

Young (1998) suggests five types of Internet addiction: (a) cyber sexual addiction, (b) cyber relationship 
addiction to online friendships or affairs, (c) net compulsions to online gambling, auctions or obsessive 
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trading, (d) information overload to compulsive web surfing or databases searches, and (e) computer 
addiction to game playing or programming. 

When properly used, Internet is an important technology that provides people with vital skills for the 21st 
century such as information access, problem solving, and self-directed learning. However, when Internet is 
used unconsciously, it can cause anxiety or fear and negatively affect personal development (Colwell & 
Kato, 2003; Kerberg, 2005). In addition, excessive use of Internet may have harmful effects on biological, 
physiological, psychological and social development of the user (Caplan, 2002). In this context, Internet 
addiction has gradually become a serious problem. Thus, although it was not mentioned in the previous 
editions, American Psychological Association has added Internet addiction as a mental illness to the fifth 
edition of the Handbook of Diagnostic and Statistical Manual of Mental Disorders. 

In many areas of behavioral addiction, it has been debated that whether some extreme behaviors can 
really be regarded as addictions or not. Social media addiction, as a type of Internet addiction, has been 
discussed in this context. Griffiths (2013) made an important contribution to this discussion by suggesting six 
essential components to describe a behavior as addiction. These six basic components are salience, tolerance, 
mood modification, relapse, withdrawal, and conflict (Griffiths, 2013, p. 121). He states that a behavior can 
be defined as addiction if it meets these six components. 

Around the world, a number of studies have been conducted on whether the Internet and its applications 
are addictive or not. For example, in a study conducted in China which was attended by 2,620 students, it 
was identified that 24% of the participants were addicted to social media. One of the notable points here is 
that students who have developed high scores on addiction have got low scores on time management. In 
addition, internet-addicted adolescents provided meaningful results in emotional symptoms, problem 
management, hyperactivity situations and lower social behavior scores (Cao et al, 2007). 

Social network sites are virtual communities where users can create individual or public profiles, interact 
with friends in real life, and come together with other people based on common interests (Kuss & Griffiths, 
2011, p.3529). The first social networking example - Six Degrees - was published in 1997 on a basis that 
would allow everyone to connect via six-degree distance. In 2004, Facebook was established, which is now 
considered as the most successful site with about 2 billion users, more than half of the users logging every 
day. 

Nowadays, intensive use social media applications by ordinary users have been increased; and this has 
raised the concerns for addiction. Surveys consistently show that young people and students benefit most 
from social networks in the general population (Kuss & Griffiths, 2011). The rapid increase in the use of 
online networks -mainly on the basis of time spent- has led some researchers to argue that excessive social 
networking can lead to addiction (Griffiths & Pontes., 2014, p. 120; Sussman et al., 2011). Sometimes 
individuals can have a variety of actions that are potentially addictive, such as using social networks 
extensively. 

Research on social media addiction levels has produced inconclusive results because some are concerned 
about negative consequences of intensive use, while others have focused on gains through the use of social 
media. However, latest research showed that greater access to social media brings greater concerns. For many 
institutions, social media addiction has become a major concern. Davies and Cranston (2008) found that 
participants (120 managers and practitioners working on youth) were afraid that online social networking 
would replace other activities and face-to-face social interaction. When participants were asked to identify 
the risks associated with online social networks, 23% of reported addiction as a source of concern and the 
others reported exposure to bullying (53%), disclosure of personal information (35%), and sexual assault 
(22%). 

In their study, Koc and Gulyagci (2013) pointed out that Facebook addiction is related to factors such as 
usage time, social motivations, depression, anxiety, and insomnia. However, they did not find any significant 
relationships regarding demographic factors. In several studies, researchers found that frequent access to 
Facebook is associated with clinical manifestations of psychiatric disorders (Rosen et al., 2013), excessive 
use of social media can disrupt interpersonal relationships and jeopardize the quality of relationships people 
develop with others (Tokunaga, 2011), and it might reduce self-esteem (Peter & Valkenburg, 2006). 

There are many social and psychological factors that affect social media addiction. The role of parents is 
particularly emphasized by researchers. It has been reported that while the lack of family love (Huang & 
Leung, 2009) and the conflict between parents and children (Yen, Ko, & Yen, 2007) may lead to dependent 
behaviors; supportive parental supervision (Lin, Ko, & Wu, 2009) decreases impact on dependent behaviors. 
Many other variables, such as the need to establish a close relationship, narcissistic behavior, excess of 
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leisure time, and the limitations of the social environment, may be determinants of social media addiction. At 
the same time, negative effects on adolescents can sometimes be reflected in course performance and social 
capital. 

In short, social media addiction is an ever-increasing problem in the 21st century. For this reason, a 
number of studies were conducted in many countries on social media addiction. Each study presents a new 
outcome, explains reasons and effects of the social media addiction, and presents new ways of solutions. 
However, the problem is getting bigger every day and public authorities, educators, and parents do not really 
know how to overcome this challenge. This gap sets the background of the present study. 

The main purpose of this study is to identify the social media addiction level of high school and 
university students based on internally-recognized measures and propose solutions for reducing the degree of 
the problem. Toward this main purpose, the following research questions were investigated: 

• What is the social media addiction level of high school and university students based on 
internationally-recognized measures? 

• Are there significant differences between social media addiction levels of high school and university 
students in general and in terms of sub-categories of technology addiction? 

• Are there significant differences in social media addiction levels of students in terms of gender, school 
type, department, and duration of daily use? 

• Is social media addiction level of Turkish students different from the social media addiction levels of 
students in other countries? 

• What kinds of measures should be taken to reduce social media addiction levels of high school and 
university students in Turkey? 


2. METHODS 

2.1 Participants 

The literature usually suggests that majority of social media users are teenagers and young adults. In practical 
terms, high school students are teenagers and university students are young adults. Considering that most of 
the social media users fall within these two categories, their age groups make them potentially more 
vulnerable for social media addiction compared to other segments of the population. For this reason, the 
sample of the present study was chosen from among high school and university students. 

The sample of the study consisted of 700 students. Of them, 345 (49%) were female and 355 (51%) were 
male. Gender distribution was almost equal. As far as their schools are concerned, 397 (57%) of the 
participants were high school students and 303 (43%) participants were university students. High schools 
students formed the majority with a difference of 14%. Regarding the amount of time they spend on social 
media per day, 115 (16%) used social media for less than an hour, 344 (49%) used for 2-3 hours, 145 (21%) 
used for 4-5 hours, and 96 (14%) used for more than 5 hours. Approximately 85% of students use social 
media for more than 2 hours per day. High level of a mobile phone ownership (96%) is likely to increase 
daily use of social media. 

The composition of the high school sub-sample is generally similar to that of the total sample, partly due 
to the fact that it formed the majority. Within the high school sub-sample, 48% were female and 52% were 
male; 48% were in Anatolian High School and 52% were in Science High School; 84% use social media for 
more than 2 hours a day; and 96% have their own mobile phone. 

The composition of the university sub-sample is a little bit different. Within this group, 46% were female 
and 54% were male; 11% were from Journalism, 24% were from Public Relations and Advertising, 55% 
were from Communication Design and Management, and 10% were from Cinema and Television; 87% use 
social media for more than 2 hours a day; and finally 99% owned a mobile phone. 

2.2 Instrumentation 

A Likert-type scale was used to collect data in this study. The scale consisted of two parts. In the first part, 
participants' personal information regarding their gender, school, department, duration of daily use, and 
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ownership of mobile phone were asked through structured items. In the second part “Bergen Facebook 
Addiction Scale” (BFAS), which was developed by Andreassen, Torsheim, Brunborg, and Pallesen (2014), 
was used to collect data about social media addiction of the participants. The fact that BFAS has been used in 
many international studies has allowed researchers to compare the results of the present study with the results 
of similar studies in other countries. 

The validation process of the scale has taken place in several phases. First, all the 18 items in BFAS were 
translated into Turkish. Because the original scale was about Facebook addiction, the items were adapted to 
social media addiction. Secondly, language experts and social media specialists reviewed the translated 
version of the scale. Third, following several minor revisions, the draft was pilot-tested with 25 high school 
and 25 university students. A few revisions regarding the wording of the items were made. Finally, based on 
the obtained data, Cronbach Alpha reliability coefficient of the scale was calculated as .89 (highly reliable). 

The participants' mean scores were used to determine both how intensely each of the symptoms of 
addiction is experienced by individuals, and the level of social media addiction they experience. Having a 
total of 18 five-point items, mean scores up to 20 (1.00-1.15 over 5 points) were used to describe a natural 
use, 21-39 (1.16- 2.21 over 5 points) mild addiction, 40-69 (2.22-3.87 over 5 points) moderate addiction, and 
70-90 

(3.88-5.00 over 5 points) severe addiction. 

BFAS has been used to collect data in a number of countries and each study set the cut-off values for 
different levels of addiction. De Cook et al. (2014) in Belgium used the mean score of 2.00, Wang et al. 
(2015) in China used 3.00, and Jaffri (2015) in Bhutan used 4.00 as the critical score differentiating 
non-addicted and addicted users. On the other hand. Salem et al. (2016) in Egypt used the following score 
ranges to differentiate addiction levels: 0-20 (natural use), 20-39 (mild addiction), 40-69 (moderate 
addiction), and 70-90 (severe addiction). The present study assumed that these four layers may serve as better 
indicators of addiction levels compared to a rigid dichotomous split as non-addicted and addicted. 

2.3 Procedures 

Data collection procedures were generally similar in both sub-samples. After obtaining necessary 
permissions from the selected schools, the researchers provided elaborated information to onsite instructors 
to administer the scale. While gathering data from high school students, the principal and a guidance teacher 
collaborated with the researchers. They administered the scale to their students as a part of guidance and 
counseling activities in the school. It took approximately four weeks in March 2017 to complete data 
collection in the high school. In the university setting, two professors administered the scale to 
communication students in compulsory courses. The administration of the scale to university students also 
took about four weeks during April 2017. All the students were informed about details of the study, 
participation was voluntary, and no rewards were given to the respondents for their attendance. 


3. FINDINGS 
3.1 Gender 

There were 18 items on the scale and possible responses for each item ranged from 1 to 5. Therefore, the 
minimum possible score was 18 and the maximum possible score was 90. Table 1 shows descriptive statistics 
with regard to gender. 

Table 1. Results of t-test Regarding Gender Effects for All Participants 


Addiction 

Male (n=355) 



Female (n=345) 



General (n=700) 

Scores 

Mean SD 

Min 

Max 

Mean 

SD 

Min 

Max 

Mean 

SD p 


2.35 0.73 

1.0 

5.0 

2.54 

0.80 

1.0 

5.0 

2.44 

0.76 .001 
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The mean addiction score of the total sample is at the moderate level (M=2,44). This figure is neither too 
high nor too low but it deserves some attention. There was a meaningful difference between total addiction 
scores of the participants according to their genders. As seen in Table 2, the mean score of female students 
(M=2,54) is higher than the mean score of male students (M=2,35). The difference between these two scores 
is significant (p<.001). It is possible to say that female students are more addicted on social media than male 
students. 

When considering the high school students, there are gender differences on some dimensions of social 
media addiction. There is no significant differences between male and female students when we look at the 
dimensions of salience (M=2,50 vs. M=2,63; p=.163), tolerance (M=2,67 vs. M=2,80; p=.214), relapse 
(M=2,34 vs. M=2,51; p= .138), and conflict (M=2,30 vs. M=2,42; p=.216). There is, however, a significant 
gender effect on mood modification (M=2,26 vs. M=2,61; p<.005) and withdrawal (M=2,35 vs. M=2,66; 
p<.008) dimensions. Mean scores of female high school students regarding the dimensions of both mood 
modification (M=2.61) and withdrawal (M=2.66) were significantly higher than the mean scores of male 
students (M=2.26 for mood modification; M=2.35 for withdrawal). Yet both male participants (M=2,39) and 
female participants (M=2,61) appear to have moderate addiction levels so that the overall level of addiction 
of participants was moderate (M=2,51). 

Findings for university students has some similarities as well as differences. There was no statistically 
significant differences between male and female participants in terms of salience (M=2.48 vs. M=2.58; 
p=.335), tolerance (M=2.68 vs. M=2.89; p=.095), relapse (M=1.74 vs. M=1.94; p=.090), withdrawal 
(M=2.60 vs. M=2.58 p=.899), and conflict (M=2.01 vs. M=2.11; p=.411). When the mean scores of mood 
modification with regard to gender are compared, the mean score of female students is statistically higher 
than the mean score of male students (M=2.15 vs. M= 2.58; p<.001). In this case, it is possible to say that 
female students are more addicted in terms of mood modification so they experience more emotional changes 
in their interactions. 

3.2 School Type 

Table 2 presents the findings regarding the addiction levels of high school and university students. As seen in 
the table, the mean addiction score of high school students (M=2.51) is higher than the mean addiction score 
of university students (M=2.36). The difference between the mean score of high school students and the 
mean score of university students was statistically significant (p<.013). Therefore, it can be said that high 
school students are more addicted to social media compared to university students. 

Table 2. Results of t-test for Comparisons of High School and University Students 


Addiction 

University (n=303) 


High School (n=397) 


General (n=700) 

Scores 

Mean 

SD Min 

Max 

Mean SD Min 

Max 

Mean 

SD p 


2.36 

0.76 1.00 

5.00 

2.51 0.77 1.00 

5.00 

2.43 

0.76 .013 


As mentioned before, the high school sub-sample included two types of high schools: Anatolian High 
School and Science High School. Further analyses showed that there was no significant difference between 
high school types with regard to salience (M=2,55 vs. M=2,58; p=.707), tolerance (M=2,79 vs. M=2,68; 
p=.311), mood modification (M=2,44 vs. M=2,44; p=.989), relapse (M=2,40 vs. M=2,46; p=.623), conflict 
(M= 2,35 vs. M=2,37; p=.875). There was a meaningful difference only in the withdrawal dimension 
(M=2.39 vs. M=2.64; p<.034). Thus, it is possible to say that Anatolian high school students who started to 
the school with relatively low entry scores and who are less successful in the process than Science high 
school students, who have better entry scores and higher academic performance during the courses 
experience more difficulties to leave social media when they wanted. 

Department-based differences were examined for university students. There was a statistically significant 
departmental effect regarding the withdrawal dimension only. According to the outcomes of the follow-up 
test, students in the Journalism Department had lower addiction scores than their counterparts in the 
Department of Public Relations and Advertising (p<.004) and the Department of Communication Design and 
Management (p<.020). No significant differences were found in other dimensions of addiction. 
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3.3 Duration of Daily Use 

As indicated before, approximately 85% of students use social media for more than 2 hours per day. Situation 
is almost the same for high school and university students. Therefore, the impact of frequent or longer use of 
social media on addictive behaviors were examined. The results of ANOVA are exhibited in Table 3. It is 
clearly seen in table that duration of use (amount of daily time) has a significant impact on social media 
addiction (pc.OOO). 


Table 3. ANOVA Results for Durations of Use 



Sum of Squares 

df 

Mean Square 

F 

Sig. 

Between Groups 

92,982 

3 

30,994 

66,197 

,000 

Within Groups 

325.870 

696 

,468 



Total 

418.851 

699 





After ANOVA test, multiple comparisons were made for varying amount of daily time spent on social 
media (ranging from less than an hour to more than five hours per day). All comparisons produced significant 
results (p<.000). The biggest difference was observed between the minimum amount of time to maximum 
amount of time (mean difference was 1,21/5,00). It means, when the time spent for the use of social media 
increases, the level of addiction also increases. Stated alternatively, the lesser the time, lower the addiction on 
social media. Further analyses showed that more exposure to social media increase the level of addiction in 
all dimensions. 

This is due to the fact that the correlation coefficients among dimensions of social media addiction are all 
positive and significant. The lowest correlation is between salience and mood modification (r=.287), while 
the highest correlation is between relapse and conflict (r=.609). The correlations among dimensions of social 
media addiction are at medium or high levels. When separate calculations were made for high school and 
university students, the general situation was similar for both groups. The lowest correlation was is between 
salience and mood modification (r=.231 for high school and r=.296 for university students), while the highest 
correlation is between relapse and conflict (r=.600 for high school students and r=.595 for university 
students). 


4. CONCLUSIONS AND RECOMMENDATIONS 

Students in the present study reported a moderate level of social media addiction. This suggests that if the 
right steps are taken, it will be possible in the future to overcome this type of addiction in Turkey. Gender 
played a significant role in this regard. It appears that female students are more addicted on social media than 
male students. When considering high school students, there are gender differences on the dimensions of 
mood modification and withdrawal but other differences are not statistically significant. Findings for 
university students have some similarities as well as differences. There was no significant differences 
between male and female university students in terms of salience tolerance, relapse, withdrawal, and conflict. 
However, the gender difference for mood modification was significant, females being more addicted. Gender 
was found as an important factor affecting social media addiction by Steggink (2015) in the Netherlands and 
by Monads et al. (2017) in Italy. On the other hand, there is no relationship between gender and social media 
addiction on the results of Wang et al. (2015) in China, Jafarkarimi et al. (2016) in Malaysia, and Blachnio et 
al. (2016) in Poland. This leads to a cross-cultural interpretation of the fact that gender has an impact on 
social media in some countries. Therefore, each culture should be assessed in a deeper way within its own 
specific conditions. 

It is found that social media addiction level of high school students is significantly higher than social 
media addiction level of university students. It can be claimed that the factors such as characteristics of age 
group, educational status, and intensified text-anxiety (even stress and depression) regarding the university 
entrance exam that the Turkish high school students confront may be effective. This result is supported by 
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Zaffari's (2015) and Wang’s (2015) studies revealing that that social media addiction is related to depression 
and anxiety of users; that is, as the levels of depression and anxiety increase, the users become more inclined 
towards social media addiction. Thus, it may be helpful for researchers to seek elaborate information on this 
issue. 

Anatolian high school students stay more on social media than Science high school students. Science high 
school students have a more demanding curriculum, and students who study in these schools concentrate 
more on their courses; it is probably effective in low addiction scores. Similarly, when we look at the 
departmental effects on social media addiction, academic department in which students pursue an 
undergraduate degree has an impact on salience and withdrawal but not on other dimensions. Indeed, 
according to Zaffari et al. (2015) study on Pakistani students, there is a negative relationship between 
academic performance and Facebook addiction. It will be useful for researchers to investigate this issue 
further in the future studies. 

The present study determined that both the majority of high school students (84%) and that of university 
students (87%) use social media for at least two hours a day. However, as the duration of social media use 
increases, the level of addiction (together with all dimensions) increases. This situation can be explained with 
the increase in daily time spent on social media and the increase of the connection with the virtual life that 
the individual has established there. The addiction study of Jaffarkarimi et al. (2016) on students in Malaysia 
also shows that time of use is an important variable affecting addiction. 

The comparison of the social media addiction level of students in Turkey with different countries will be 
helpful in identifying where Turkey stands. In a study conducted by Salem et al. (2016) in Egypt and Kuwait, 
participants' addiction levels were reported as 49.5 over 90 on average. In addition, the studies in Belgium 
and Germany show that addiction in these countries is not yet fully developed. According to Idubor's work in 
Nigeria, there is a high social media addiction among students in this country (Idubor, 2015). Finally, 
according to the study by Zhang et al. (2008) comparing the addiction level of young people in USA and 
China, 14% of students were heavily addicted to the Internet, 64% were slightly addicted, and 22% were not 
addicted. Among the U.S. respondents, 4% were heavily addicted to the Internet, 23% were slightly addicted, 
and 73% were not addicted. These figures show that social media addiction is a phenomenon that varies from 
country to country. Turkey places somewhere in the middle with moderately disturbing signs. This can be 
explained by variables such as countries' cultures, education systems, social structures, practical 
opportunities, and the relationship with technology. It would be useful to carry out research that compare 
cross-cultural social media addiction so that the causes of this situation can be better understood. 

As a final remark, the use of social media is increasing at a very rapid pace in Turkey as the case in many 
other countries. This situation brings the risk of social media addiction, especially among young people. In 
order to proceed further in a healthy manner, effective measures should be taken in social, political, cultural, 
and educational contexts. Pertinent actions should focus particularly on improving legal regulations, 
educational opportunities, public awareness, psychological services, media consciousness, technological 
skills, parental guidance, and individual attentiveness. 
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ABSTRACT 

University students in Macao are required to attend computer literacy courses to raise their basic skills levels and 
knowledge as part of their literacy foundation. Still, teachers frequently complain about the weak IT skills of many 
students, suggesting that most of them may not be benefiting sufficiently from their computer literacy courses. This 
research proposes an enhanced framework based on constructivist principles by using peer-tutoring to increase cost 
effectiveness and to improve student outcomes. Essential to this proposed model is the training of former course 
graduates as peer-instructors to achieve high quality learning results. At Institute de Formajao Turistica (IFT), a case 
study was used to evaluate its effectiveness using a qualitative analysis. In Macao, most students have a Confucian 
Heritage Cultural (CHC) background and the current findings demonstrate that students share more easily their learning 
difficulties within their group as their interpersonal relationships improve. It is suggested that since CHC cooperative 
learning is primarily based on bonds, students involved in this 'relationship-first, learning-second 1 type shared a larger 
amount of knowledge and social skills, a dual positive outcome. Moreover, English language is a major barrier for the 
understanding of the teacher’s message to Chinese students. Meanwhile, the negative Western concept of plagiarism is 
replaced, under the CHC, as the ‘face giving’ and it is directly based on the relationship intensity to help friends'. At last, 
peer-tutors play a key role in the student increase internal motivation regarding the joy of the learning process. 


KEYWORDS 

Macao, Confucian Heritage Cultural, Computer Literacy, Constructivist Principles, Peer-tutoring 


1. INTRODUCTION 

Computer literacy includes a broad range of knowledge such as how to use word processing, spreadsheets, 
presentations and database software. Internet and online activities are also components of any typical 
computer literacy syllabus. Nowadays, information technology proficiency are perceived to be critical among 
employees (Murray, Sherburn & Perez, 2007) and recognized as an essential skill in today’s competitive job 
market (Grant, Malloy & Murphy, 2009). Therefore, universities should periodically update their IT curricula 
based on the recommendations made by IT experts (Kvasny, Joshi & Trauth, 2011). Besides this dogmatic 
practice and teaching alignment to comply with benchmarks recognized worldwide by the industry (such as 
ECDL, ICDL, CLAIT, Microsoft Digital Literacy Standard Curriculum and ISTE), there is also a need for a 
reshape of the education sector to nurture critical thinkers (Oliveira, Negreiros & Neves, 2015). 

In Macao, there is no common curriculum for computer literacy courses. Different schools teach them 
based on their own established traditions within their institutional culture. Chan (1999) reported that Macao 
teachers typically use the traditional didactic approach to their students, so teaching staff and institutions can 
easily claim that undergraduates have acquired plenty of skills. This didactic approach is a teaching strategy 
in which apprentices are saturated with knowledge, content and exam requirements to strengthen their 
abilities in answering exam questions. Meanwhile, the results of 2012 and 2015 PISA tests (Program for 
International Student Assessment, a worldwide study by the Organization for Economic Co-operation and 
Development in member and non-member nations of 15-year-old school pupils' scholastic performance on 
mathematics, science and reading) show that students from Asian countries have the ability to score high 
(Kennedy, 2016) and they hold a CHC background. 
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In order to pass the exam, students generally rely on rote learning and repetitive practice - memorizing 
detailed actions to fulfill tasks. With this approach, students feel tired and reluctant to learn (Chan, 1999) 
and, hence, they can easily be cognitively overloaded (Cheung, 2006). There is a "high score, low ability" 
phenomenon, which is common among CHC learners (Wong & Wong, 2016). 

At IFT of Macao, teachers use this didactic approach within computer classes. This includes written tests 
involving software exercises. Boolean concepts and decision-making skills related to this computer course. 
As learners usually perform well in these quizzes, educators believe that they have learnt the computer 
literacy proficiency and are prepared for the real world. 

ISTE (2000) suggested that, although technology education has allowed students to gain computer skills, 
the effectiveness of learning is limited. A possible reason for this outcome is the fact that computer literacy 
training (CLT) is often not aligned with elements of realistic and authentic application. Even if students have 
this knowledge, they do not know how to transfer it to real life (Cheng, 2005). Chan (1999) reported that this 
authentic application of computer knowledge should be an objective for students but this target population 
does not seem to have met this goal. Chan (1999) also commented that the current learning approach 
typically used in Macao is not suitable for the fast-paced world of information technology of today. 

Quite often, classes have forty or more students, so very little individual attention can be given to each 
single student to address his/her problems. One way of providing this high level of individual teaching 
cost-effectively is through the introduction of peer-tutors, which involves successful graduates from previous 
courses as instructors in successive cohorts of students. This approach would significantly reduce the 
teacher’s workload by employing less expensive teaching staff. Moreover, the opportunity to pursue a 
teaching career in the future would be a strong incentive for students to work hard and become successful 
peer-tutors. After all, peer-tutoring allows the development of critical thinking and improve confidence and 
interpersonal skills (Wong, Wong, 2016). 

Taking into consideration this alternative teaching environment. Figure 1 shows that teaching staff are not 
necessarily the only source of learning and students are no longer the only source of feedback. With the 
introduction of peer-tutors, these could provide assistance to undergraduates and feedback to lecturers. 
Besides, English (as the official medium at IFT) can also be considered a communication barrier for Chinese 
speakers, particularly with fresh students, allowing peer-tutors on the Chinese-English conversion help. 
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Figure I. The Teacher as a Facilitator of Knowledge 

The present research proposes an alternative classroom management to accommodate the specific needs 
of universities in Macao, in general, and IFT, in particular. Furthermore, it will add to our current knowledge 
the sense of developing and maintaining peer-tutor groups and validate ways of establishing small 
cooperative groups of students and tutors, who assist them and help the professor in monitoring the 
classroom. Finally, it allows learners to learn within teams while enabling professors to facilitate teaching 
and peer-tutors to engage into group discussions. 
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This article is divided into further five sections. The second one introduces the research questions, design 
and methodology of the present study while the pilot study is covered in section three. The cycle phases one 
and two are explored on the following two sections. Section six presents the first inferences, including the 
analysis of the research questions and its implications to the CHC environment. 


2. RESEARCH QUESTIONS, DESIGN AND METHODOLOGY 

Over the past decade, there have been an increase number of secondary school graduates entering higher 
education, particularly, in Macao. Many studies of cooperative learning and its effects on student 
achievements have taken place at various grade levels and in several types of educational settings in 
numerous countries (De Smet, Van Keer & Valcke, 2008). Peer-tutoring is also used by many teachers from 
as early as pre-school education up to the tertiary level (Slavin et al., 2003). Indeed, research shows that 
cooperative learning is an effective way for students to maximize their own learning and the academic 
accomplishments of their peers (Slavin et al., 2003). Furthermore, it allows students to boost their academic 
performance as well as their social-emotional skills (Davison, Galbraith & McQueen, 2008). Openness has 
been connected to creativity in the workplace, for instance (ETS, 2016). 

This peer-tutoring program is innovative within IFT. Although the results of these numerous studies have 
been published, the combination of these two methods and its application in Macao are a novelty. It is also 
new to most undergraduates of Macao tertiary institutions who have spent their last 12 years studying under 
the traditional Chinese setting. Certainly, little research appears to have been written on what factors 
influence the learning growth in Macao. 

The aim of this research is to analyze whether or not computer literacy training for CHC students can be 
improved but without a major increase of the financial cost of delivery. Together, peer-tutoring and 
constructivist principles offer a possible answer to this issue, which can be broken down into the following 
three research sub-questions: (1) To what extent might constructivist principles contribute to an improvement 
of the quality of teaching and learning in computer literacy courses? (2) To what extent might the effective 
use of peer-tutoring improve the quality of teaching and learning in computer literacy courses without 
significantly increasing the cost of provision? (3) To what extent might peer-tutoring based on constructivist 
principles improve the levels of student stimulus for learning in Macao? 

The methodology consists of an intervention based on qualitative data. Therefore, it will be conducted 
within the so-called action research paradigm and based on the following five items: 

1. Questionnaires will be used to gather data concerning affective issues of working in teams. Both 
closed and open questions will be included in order to verify the improvement process. 

2. Participant observation will take form in the researcher’s participation as a viewer, where the 
researcher’s identity as an investigator will be openly disclosed. 

3. Students’ journals will be used to survey respondents' opinions, attitudes and views. 

4. Semi-structured interviews will be held with focus groups to provide data on how people think and 
feel about the topics concerned in this research. 

5. Grading of assignments. 

2.1 The Particular Role of the Researcher and Peer-tutors 

As said, the first researcher took the teacher's role in the class and as a participant observer, which means 
students were aware of the research project he was carrying out. As relationships were being developed 
between participants, perceptions and feelings on this teaching were shared among the different stakeholders. 
For reference, a total number of 130 students took part of this study. 

2.2 Criteria for Selecting Peer-tutors 

Two kinds of peer-tutors were required: senior peer-tutors (SPT) and junior peer-tutors (JPT). The first ones 
were selected from the pool of the second, third and fourth year students who had already taken this course 
successfully. JPTs were selected just before the start of the action research and based on their ability to lead 
each group in tackling problems within the class. The criteria for selecting senior peer-tutors were as follows: 
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(1) Peer-tutors must hold a Merit “B” level in the previous computer literacy course. (2) They must 
demonstrate confidence in teaching the subject. (3) They must have sufficient time available for 
peer-tutoring. 

Twenty peer-tutors showed but only four were available to participate in the present experience. 
Afterwards, a brief training was given to them, mainly focused on the introduction of peer-tutoring concept, 
topics to teach and problems that they will encounter when giving guidance to students. Questionnaires were 
distributed to senior undergraduates (potential JPTs) in order to identify, allocate and verify problems. 

This computer literacy courses in IFT consist of two modules: Computer Application I (MS-Word. 
MS-Excel, MS-PowerPoint, MS-Movie Maker, Adobe Photoshop) and Computer Application II (Ecommerce 
construction site with Adobe Dreamweaver and Wix.com). Their learning aims are to equip students with the 
ability to handle IT situations which typically occur under the tourism and hospitality industries. These 
courses are compulsory for all first year students of IFT. 


3. PILOT STUDY 

The pilot-study was carried out between September and October 2014 with students for whom Chinese is 
their native language. Unwillingness to learn and a general feeling of boredom were, broadly speaking, the 
overall rule. Conceptually, peer-tutoring (system of using students to tutor other students), information 
(teacher's slides, handouts, recommended textbooks and web pages), scaffolding (process in which teachers 
model or demonstrate how to solve a problem, and then step back, offering support as needed) and learning 
via interpersonal relationships are possibilities of learning strategies. This last category entails a pre-requisite: 
the building of bonds within the classroom. There is a strong agreement that a basic trust relationship is one 
of the foundations for effective learning. As a result, a few students reported that neither they asked nor hold 
any form of communication with SPTs. Fortunately, all SPTs worked hard on building friendship 
connections by initiating conversations with students and actively helping them to overcome difficulties, 
thereby building a trusting connection. 

3.1 First Inferences 

The available qualitative data identify some issues encountered by learners during this computer literacy 
course: 

1. Students rely heavily on using their mother language (non-English) as their sole learning medium as 
opposed of using English as the official teaching medium (most of the respondents favor being taught in their 
native dialect). 

2. It is essential that skills and knowledge learnt during the computer literacy courses are applicable to 
real life circumstances. 

3. As expected, the increase of the students’ number requires a significant increase of staff and 
resources within limited budgets. 


4. ACTION RESEARCH - CYCLE 1 

In the first day, the teacher explicitly conveyed the course relevance to everyday problems and the 
development of future professional careers, the hands-on perspective. Background questionnaires were 
distributed to students concerning knowledge of computers, video making and presentation skills. SPTs and 
JPTs were introduced to the teaching approach, roles to play within the classroom and the assessment 
requirements. For each learning item written on the whiteboard, the professor urged SPTs to guide students 
with examples of real life cases. 

Some pupils confirmed that although they did not contemplate themselves as experts on the subject, they 
considered themselves to be reasonably proficient in handling software. In fact, some students claimed that 
they had programming knowledge, although this matter was not contemplated during this module. Others felt 
the syllabus was of little demand and, thus, asked whether they could drop the course or get exempted 
because they perceived it as a waste of time, as the following quotations depicted in Table 1 illustrates. 
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Table 1. Students’ Feedback at Research Cycle 1 

“Why am I learning this? I have learnt it since primary school!” - Student (1) _ 

“Do I have to take this course? It seems a waste of time for me to study Word, Excel and PowerPoint. I 

can even do programming with Java, Visual Basic and C++!”- Student (3) _ 

“My school required me to focus on how to create nice graphics, how to submit homework online and 
even leam how to do programming, which I didn’t have a clue of what the heck that was (even now). But 
whenever my friends and I stand in front of a computer shop thinking of buying a notebook, we felt 
computer ignorant.” - Student (9) _ 


5. ACTION RESEARCH - CYCLE 2 

From the lessons learnt from Cycle 1, all participants were invited for an informal lunch in IFT. According to 
the Chinese culture, banquets are central for developing relationships ("guanxi") or connections. This is a 
common and well known practice in the Chinese community. Meanwhile, the teacher changed the students' 
learning group size. This allowed the participants to work with smaller groups where the leader from each 
team was appointed as JPT. SPTs helped in group selection and discussed the distribution of their 
responsibilities among groups. A stronger emphasis was put in place on using a relationship approach based 
on quality of improvements among the students. This technique recognizes the bi-directional nature of bonds 
(Rose-Krasnor, 1997), that is, confidence promotion and self-respect, established effective contacts and 
taught students how to work between groups and independently (Pearson, 1999). 

In Cycle 2, a different assignment was required: production of a short video clip. Although any topic 
related to a specific aspect of Macao was suggested, students were free to choose their own type of video 
(documentary, historical, comical, love-story...). This allowed a tremendous freedom for stimulating their 
creative thinking. The professor allowed them to remain on campus during class hours to complete their task 
with their group and the corresponding JPT and SPT. Other than that, they were allowed to use their own 
personal computers so everything they accomplished became a personal experience for and by them. 

Similar to the first cycle, JPTs and SPTs held a briefing before the start of this cycle. Both JPTs and SPTs 
strove to maintain harmony (Gabrenya & Flwang, 1996) and a positive peer-relationship (Lee et al., 2004) 
with the students. JPTs' initial tasks were to keep students working on the project, which involved selecting a 
video topic, title, storyline, casting, editing and assignment preparation for submission. Through discussions, 
SPTs and JPTs used their own expertise to solve on-going difficulties. Using conflict resolution strategies, 
SPTs and JPTs were able to address positive outcomes or mitigate relationships damage, which, in turn, 
could have affected the learning motivation factor. 

Students' journals contained written notes about emails and online forums or verbal comments made 
outside of the class time about the different topics of the subject. Daily entries were classified under the 
following three leadings: (a) Evaluate your friendship with your group mates including SPTs; (B) Reflection 
on learning goals; (C) Evaluate your learning. When compared with their first remarks of Cycle 1, the 
amount of feedback had increased from 13 to 25 and a major pattern could be found: the shift from extrinsic 
to intrinsic motivation. 


Table 2. Students’ Feedback at Research Cycle 2 

“Students-teacher is helpful and cool! The knowledge is useful. He told us how he uses the software on 
his work and life. It seems quite important." (Student 29) 

"Please, ask senior peer-tutors to teach other topic." (Student 6) 

“Kindly, find more helpers. We need them.” (Student 14) 


As illustrated on Table 2, students highly appreciated the guidance from SPTs. As well, students reflected 
on how different software may fit their life-styles: “Respectful teacher, I was brought up in a Mainland 
Chinese community and I have learnt a lot about the need to honor and obey parents' expectations. I didn't 
want to come here to study. At first, my only wish was to take this degree to honor my dad. My relatives 
have already planned my future after my graduation and I know I will get a good job. So, I just need to get 
high marks, go back with my first honored degree and nothing else. After I’ve known my classmates better, 
especially SPTs, and through this share of experiences, my horizon has been broaden by knowing how people 
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work outside of China and how to use different software for my everyday life (and not just to get marks). 
Studying here became more enjoyable and meaningful." (Student 18) 

This quotation highlights an aspect of the traditional Confucian Heritage Culture: The intrinsic interest of 
the student learning was enhanced by three factors, that is, SPTs' encouragement, collaborative learning with 
classmates and the relationship among the participants (including the teacher). In fact, some students even 
admitted that they should have this teaching approach in all IFT classes. However, two students worried that 
their lack of experience might have weakened the group’s overall performance, which could have damaged 
their relationships with other students during their team work. 

SPTs also participated in the video production by guiding them through the process of thinking, planning 
and acting and on the hands-on experience. Meanwhile, JPTs were busy on administrative work, i.e., 
arranging meetings, deciding discussion topics and showing students how to proceed, especially in editing 
videos. 


Table 3. Added Comments of Students 

"I taught them how to edit the video for their personal work. One student had problems with the software in his 
computer, so I helped him. I edited some of the project videos while the remaining colleagues did other stuff." 
(Student 11, JPT) _ 

"I have done my personal tutorial with the student teacher's help because our group had only five people. So 
everyone had to do something to help to finish the job. It was fun. We became good friends." (Student 16) 


In this cycle, the students expressed their feelings more openly. Their emotions ranged from initial 
concern to increased interest. The pupils that had shown signs of anxiety replied to the teacher's email with 
encouraging words: "Last night, I watched my own video with my group and I nearly cried of happiness. I 
never thought I could make a video so nice. The group leader and others helped me very much. I made many 
mistakes but they forgave me with jokes." Other students placed high value on the quality of their 
relationships: "I don't want the SPT to leave. I missed them. They had been extremely kind and helpful to our 
learning process. I hope that they can be with us in the next semester." However, one particular student 
worried about the future: "I am sorry sir, the class was great. I did fine. Thanks. But will you be here teaching 
next year? Will the senior peer-tutors be here with us? I am worried that if they don't come I may not do 
well." (Student 22). 


6. CONCLUSIONS 

6.1 Research Questions 

6.1.1 To What Extent Might Constructivist Principles Contribute to an Improvement in the 
Quality of Teaching and Learning in Computer Literacy Courses? 

This program of study pioneered an enhanced one in Macao where students no longer mainly gain knowledge 
and experience through the professor’s direct influence or guidance in class, but through using their own 
experience developed in partnership with peer-tutors, teachers and classmates. The traditional learning 
approach involves students learning content by examinations, presentations and quizzes, mainly. This often 
reduces their motivation to learn, although there is evidence that students hold a higher motivation if their 
learning is based on authentic experiences of real life issues and shared ideas within their group. Therefore, 
students receive more interpersonal experience from the group members (who they had learnt to trust and 
with whom they had built new learning relationships). This happens because of the CHC group dynamics, 
whereas the emphasis was put on the student activity and cooperative work. With highly collaborative 
efforts, stronger motivation and incentive for learning, students achieve higher course results. Moreover, 
when cooperative work is based on relationships among Chinese students, their collaborative assistance is 
sustained throughout the course. 
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6.1.2 To What Extent Might the Effective Use of Peer-Tutoring Improve the Quality of 
Teaching and Learning in Computer Literacy Courses Without Significantly Increasing the 
Cost of Provision? 

With the rapid development of Macao, an increasing demand for computer literacy skills training, staff and 
hardware can be expected. To control this situation, two components should be stressed: 

a) Computers and their peripherals: These require constantly updated software as well as the latest 
hardware state-of-the art, which always add significant costs to any institution’s budget. Yet, students can use 
their own laptops instead. Hence, no additional cost is accrued by the university with the exception of 
printers and Internet access, for instance. 

b) Teaching staff: The increasing number of students would require the hiring of more teachers. 
Besides, they reported that they were unable to adequately meet the individual needs of every student in 
class, despite their hard work. Using the peer-tutoring strategy advocated here, universities gain the freedom 
to hire tutors and allow, at the same time, more flexibly for professors to do research and work in additional 
university projects. 

6.1.3 To What Extent Might Peer-Tutoring Based on Constructivist Principles Improve the 
Levels of Student Motivation for Learning? 

According to this experience, this index has increased significantly (although it was not measured 
quantitatively) with an intensive shift from a surface to a deeper approach towards content knowledge, 
revealing that students became more interested on the subject and by being able to relate computer literacy 
content to their personal needs. 

6.2 Implications on Teaching Practice 

Under the CHC setting, teaching requires a professor that understands the need for inter-personal 
relationships, which should be firmly established before cooperative learning takes place. Additionally, these 
bonds should be developed continuously, requiring teachers to be aware of how effective interpersonal 
contacts can be encouraged and sustained within the class. According to Stankov (2010), CHC is an 
unforgiving culture, where only those who work hard deserve the highest marks attained in examinations. 
These findings advocate that CHC educators should exhibit similar behaviors towards their students. In fact, 
evidence gathered during the present study suggest that CHC teachers regard students' lack-of-response as a 
student’s own problem and not a result of weaknesses in their own pedagogy. These lecturers believe that 
students who work harder are willing to ask questions (both in-class or after class), thus, deserving better 
results. They do not appreciate the fact that, without having a first established inter-personal relationship, 
undergraduates are quite reluctant to start expressing their learning difficulties. 

The current findings also demonstrate that students share more easily their learning difficulties within 
their group as their interpersonal relationships improve in a beneficial way. It is suggested that since CHC 
cooperative learning is primarily based on relationships, students involved in this 'relationship-first, 
learning-second’ type shared a larger amount of knowledge and social skills. Particularly, when some bonds 
reach a close level and one of the participants holds a low knowledge and skill rank, the other party is more 
likely to do the work for him/her, allowing the weaker student to copy the work from the strongest in an 
attempt to extend that familiar relationship. This action, under the CHC, is usually known as the ‘face giving’ 
and is directly based on the bond intensity. However, from a western point of view, such action is known as 
plagiarism. If plagiarism holds a negative connection, the disguised 'helping of friends’ represents a positive 
one, a distinction which has characterized CHC society since ever. According to the CHB requirement, 'face' 
must be well kept especially when in front of an outsider. So, ‘your best friends must help you if you are in 
real trouble’, 

Yet, the distinction between Meng’s ideas, a Chinese Master student in Engineering in the UK, was stated 
by Gu and Brooks (2008) regarding plagiarism: “I am not quite clear where to draw a line to distinguish my 
ideas from other authors’ ideas. For example, I read a book and had some ideas of my own. So when I write, 
maybe half of the sentence is a summary of the author’s ideas whilst the other half is about my new ideas 
drawing upon the ideas in the book. Should I make a reference to the book? It was a real headache for me 
because I felt that actually my ideas were integrated with someone else’s. At present I do indicate the original 
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sources in my essays. But I feel that it seems that my own new ideas have become somebody else’s. I find 
that quite difficult and don’t know what to do.” Meng’s remarks raise the question of whether a lack of 
citation should simply be considered a transgressive act. According to both authors, beneath the surface act of 
adding references to the original textual sources was his genuine attempt to demonstrate his intellectual 
contribution, as well as his commitment to academic integrity. 
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ABSTRACT 

This study utilized sensors to investigate how females and males might perform and reflect differently on their tennis 
skills in a beginner class. A quasi-experimental design was conducted in this study to investigate the learning outcome of 
using tennis sensors. Two classes of students enrolled in a college physical education course participated in this study. 
One class of the students, who applied tennis sensor and reflective strategy, served as the experimental group; the other 
class using traditional instructional method served as the control group. The end-of-semester achievement test results 
showed that students in the experimental group performed as well as those in the control group. However, the 
experimental group performed a relatively smooth swing pattern and had better sensor-collected shot-data scores than 
those of the control group. As to gender effects, female students benefited more from using sensors when learning 
backhand groundstrokes, as compared to those not using. Male students tended to perform better on timing, swing speed, 
and spin level on making shots than those of the female students. The most common skill problems for both genders were 
maintaining balance and shifting the center of gravity of the body while hitting the balls. While viewing their own filmed 
practice videos, both gender of students reflected mostly on the key tennis techniques taught in the class; however, 
additionally, male students were more likely to refer to expectations or emotions regarding to their performance but the 
female students tended not. This study also showed that the sensors could be a useful tool in facilitating the analysis of 
tennis beginners’ performance and in assisting their learning of key techniques. Future studies should use the sensors to 
collect data from a large sample of tennis beginning players so that the differences between genders could be thoroughly 
examined and analyzed. In addition, how to improve the instructional approach while using sensors in class to better fit 
both male and female students’ needs is another critical issue in this field. 


KEYWORDS 

Sensors, Reflection, Gender, Tennis, Beginners 


1. INTRODUCTION 

Gender effects exist on student’s performance on motor skills acquisition as well as sports activities, and 
tennis is usually the case (Kramer, Rosenboim, & Shapir, 2016; Thomas & Thomas, 1988). Previous study 
showed the gender difference exists not only on novice learners but also in professional players. Take tennis, 
for example, top male players showed higher Ace, 1st and 2nd serve points won, and fastest serve speed than 
female players (Chao & Wang, 2015). What causes these difference has been investigated for a long time, 
and physical limitations, as well as the psychological tendency, are two main reasons attribute to a gender 
difference in sports learning (Vilhjalmsson & Kristjansdottir, 2003). Generally, the physical difference 
includes height, body mass index (BMI), and muscular endurance and so on; those biological factors benefit 
males in strenuous activities. In addition, males’ greater interest in sport may also affect their sport and 
physical activity involvement. Except to the difference in external performance, different psychological 
tendencies will also attribute to a diversity quality of reflections. Thus, how to identify the differences 
between genders in sport learning is critical in physical education. 

Tennis is one of students’ favorite physical education (PE) classes in college, but as it requires a high 
level of skill, such as the full-body coordination and timing of movements, beginners often find it difficult to 
master. How to improve the effectiveness of learning in group tennis class is a big challenge for college 
teachers. Feedback and modeling, are long recognized as effective approaches in motor skill acquisition 
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(Emmen, Wesseling, Bootsma, Whiting, & Van Wieringen, 1985; Hanrahan, Pedro, & Cerin, 2009; Hebert & 
Landin, 1994). Good feedbacks could provide knowledge about the correctness of movement, and enable 
learners to better clarify how to correct their movement leads to greater skill acquisition (Schmidt & 
Wrisberg, 2004). Modeling which serves as a standard of correctness helps learners to develop a cognitive 
representation and to regulate their movement, are particular benefits in the development of movement 
pattern early in learning (Magill, 1993). Thus, through demonstration of the correct technique by the 
instructor for modeling and provide verbal feedbacks according to learner’s performance was the most 
common instructional approach in PE class. However, with relatively short practice phase under the group 
learning context, lead to the difficulty to provide sufficient feedbacks for each student by one instructor, and 
usually hard to improve the learning performance effectively as well (Lee, Keh, & Magill, 1993). Thanks to 
the progress of technology, recording a video is easier than ever before, more research related to skill 
acquisition applied video feedback while learning and got positive results (Palao, Hastie, Cruz, & Ortega, 
2015; Zheng, 2013). The expert’s demonstration video could play the role of modeling, and the record of 
learners’ practice video provides visual feedback help them to better clarify their defects. Although the 
research supporting the potential benefits of video-feedback is vast, the effect of video-feedback in tennis 
learning is still questionable. Emmen et al. (1985) investigated the effects of video modeling and video 
feedback on the learning of tennis service by the novice and found no clear advantages while comparing with 
the traditional teaching approach. The possible reason might be due to the video display only provides 
knowledge of performance (movement information) but no knowledge of results (information about the 
outcome of service). In this end, learners have to notice if the ball was hit well or not through eye 
observation, lack of precise hitting data, and affect the efficiency of movement correction. In addition, the 
relatively long period of per week practicing sessions may have interfered the retention of the relevant cues 
available in videos. Thus, how to add the information of hitting outcome and extend the memory during 
non-practicing periods were the critical challenges to apply video-feedback in motor skill or tennis learning. 

Many sensors have commercially available for tennis training in this decade. Using sensors to observe 
athletes’ movements and the way they performed can provide useful information for training demands, and is 
a long-term endeavor in sports technology. The ability to detect and record each shot with precise hitting data 
can make up the lack of outcome information in traditional video-feedback instructions, in addition, the 
objective data also benefits the efficiency of learning performance analysis for educators. Apart from this, 
self-reflection can improve motor skill acquisition and help students to become more aware of their strengths 
and weakness, might be useful to extend the memory of learning (Jonlcer, Elferink-Gemser, de Roos, & 
Visscher, 2012). Most studies applied video-feedback in skill learning merely provides the instructor or 
learner’s visual information (Emmen et al., 1985; Palao et al., 2015; Zheng, 2013), and considerably less 
research has applied video together with sensor data. Besides, how the sensor data may affect students’ 
reflection process is not clear in the literatures. This study applies tennis sensors to record students’ practice 
video together with their shoot-data. The videos provided both visual information of their movement and the 
embedded shooting data (impact spot, swing speed, ball speed and level of ball spin) of each shot, which 
facilitated students’ reflection on their movement and making shots, and in turn, lead to better skill 
acquisition. 

In this study, we utilized tennis sensors to assist beginners learning basic skills and to help us collecting 
shot-data from the students for performance analysis. The aim of this study is to investigate the gender 
differences of beginning tennis players in term of their skill performance when using sensors, their ways of 
hitting of balls, as well as their reflections on performance. 


2. METHODOLOGY 

An experimental design was conducted in this study to investigate the effects of utilizing sensors to assist the 
beginning tennis players learning key skills. In addition, the sensors were used to help analyzing how male 
and female students performed differently when hitting balls, and how they reflected differently on their 
skills. The participants of this study were two intact classes of college students enrolled in a beginner tennis 
course. One class served as the experimental group, which applied tennis sensor and reflective activities in 
the class. The other class using traditional instructional approach served as the control group. After excluding 
students with high rates of absenteeism, the experimental group was comprised of 25 students (10 males, 
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15 females); the control group also numbered 25 (18 males, 7 females). Classes were held once a week for 
18 weeks, two hours per session. The instructor, course content (include instructor’s demos), teaching 
schedule, and two TAs were the same for both groups. The only difference was that students in the 
experimental group were able to view their own practice videos, and were required to answer additional two 
self-reflection questions on the online course management system (Moodle). The first question is “What 
skill-related problems have you experienced in class this week?”; the second question is “Write down the 
areas in which you feel that improvement could be made”. 

In this study, we chose the Sony Smart Tennis Sensor for our experiment (Figure 1(a)). While syncing the 
sensor with a mobile device and select the “Live Mode Video” feature on the corresponding app, it could 
record the play data together with the video ( Figure 1(b)). We use the ASUS Transformer Pad TF701T tablet 
computer to incorporate with the sensors and record both instructor’s demos and students’ practice videos. 
Each clip enables students to watch the form and shot placement in relation to their shot data. The 
achievement test was based on the Groundstroke Accuracy Assessment of the International Tennis Number 
(ITN) scoring standards (ITF, 2004). Both forehand and backhand groundstrokes performance were assessed, 
each was categorized into crosscourt and down the line shots. Five shots apiece for each of the four skills 
tested, for a total of twenty shots altogether. Each shot was assessed for accuracy, power, and stability; scores 
for each shot ranged from 0 to 7. In addition to the scores, we also record each student’s video and shot-data 
during the test in both groups, and the clips were used to analyze how students performed differently. Finally, 
a total of six weeks of reflective activities which focused primarily on the forehand and backhand shots were 
collected to investigate how males and females reflect differently. 




Figure 1. (a) A Racket attached with a Sensor; (b) A Tablet, Synced with the Sensor, is filming a Student’s Practice 


3. RESULTS 

Data collected for analysis includes students’ scores and recorded videos in the end-of-semester ITN test, and 
the experimental group’s reflection on the Moodle. 

3.1 How Females and Males Achieved Differently When Using Sensors? 

The descriptive statistics showed that students in the experimental group had higher scores compared to those 
in the control group, in addition, the male students got better performance than the female students. A 
two-way MANOVA was conducted to compare effects of group and gender on the forehand and backhand 
skills acquisition (Table 1). The analysis results showed that no significant interactive effects between group 
and gender factors in forehand skill (F (1,46) = .75, p = .393), besides, neither main effect of the group factor 
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(F (1,46) = 1.71, p =.197) nor the gender factor (F (1,46) = 1.14, p =.291) were observed. It indicated that the 
acquisition of forehand skill will not be affected by different gender, and students in experimental group 
performed as well as those in the control group. 

However, for backhand skill acquisition, the interaction of group factor and gender factor was significant 
(F (1,46) = 6.33, p <.05). To further clarify effects of gender and group in backhand skill, a simple main 
effects analysis was conducted to investigate the effect of gender between groups. The results showed that the 
effect of gender on the backhand skill is significant in control group (F (1,23) = 13.58, p <.05), however, is 
non-significant (F (1,23) = .06, p =.938) in the experimental group. It indicated that the gender difference still 
exists (Male: M=17.06; Female: M=6.26) while learning backhand skills by the traditional instruction 
approach, however, through reflective activities with watching shot-data in their videos might help to reduce 
the gap between genders (Male: M=17.70; Female: M=17.47). Another simple main effect analysis was used 
to clarify the effect of group factor between genders. The results declared that the effect of instructional 
approach on the backhand skill was significant for female students (F (1,20) = 11.53, p <.05), but is non¬ 
significant for male students (F (1,26) = .06, p =.810). It shows that female students benefit more from our 
instructional approach than the traditional methods (Experimental: M= 17.47; Control: M=6.26), but male 
students are good at both methods (Experimental: M= 17.70; Control: M=17.06). 

Table l.The MANOVA Results on Students’ Performance 



Source 

SS 

df 

MS 

F 

P 


Group 

96.79 

1 

96.79 

1.71 

.197 

Forehand 

Gender 

64.58 

1 

64.58 

1.14 

.291 

Interaction 

42.14 

1 

42.14 

.75 

.393 


Error 

2601.61 

46 

56.56 




Group 

383.04 

1 

383.04 

7.98 

.007* 

Backhand 

Gender 

331.63 

1 

331.63 

6.91 

.012* 

Interaction 

304.09 

1 

304.09 

6.33 

.015* 


Error 

2208.21 

46 

48.00 




Group 

894.38 

1 

894.38 

6.35 

.015* 

Total 

Gender 

715.20 

1 

715.20 

5.08 

.029* 

Interaction 

549.11 

1 

549.11 

3.90 

.054 


Error 

6481.87 

46 

140.91 




*p < .05 


3.2 How Females and Males Performed Differently When Making Shots? 

In order to identify the differences between genders when making shots, we analyzed students’ forms and 
shot-data in the end-of-semester ITN test. All students’ shot data and videos were recorded while attending 
the test. We randomly selected five males' and five females' videos from each group and analyze their 
performance in critical tennis techniques. In general, the key movement patterns for tennis beginners includes 
(1) proper timing for making a shot; (2) control of the wrist angle and racket angle while hitting the ball; (3) 
accurate swing trajectory; (4) maintaining balance and center of gravity when making a shot. In addition to 
this, the ability to (5) predicts the flight trajectory of an approaching ball, and (6) hit the “sweet spot” are also 
critical technical skills in tennis learning. The videos were analyzed with an experienced tennis teacher (the 
2 nd author) based on the six key techniques mentioned above. 

By analyzing the videos from both groups, we found that male students performed better on the timing for 
making a shot compared to female students. The reason for this difference may be due to male students could 
predict the flight trajectory of an approaching ball more correctly, in addition, their movement is faster than 
female students. These advantages benefit male students to perform a relatively complete swing pattern 
( Figure 2). However, the misjudgment of flight trajectory usually forced female students changed their forms 
to fit the ball’s position and affect their swing patterns as well as the timing of hitting (Figure 3). In addition, 
the major differences between male and female students on shot-data scores are the swing speed as well as 
the spin level. Such a difference could be attributed to the physical limitations that males usually have greater 
physical power than females. 
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Figure 2. A Student Hit the Ball with a Relatively Complete Swing Pattern 



Figure 3. A Student Did Not Hit a Ball with The Proper Timing 


The videos also showed the skill of maintaining balance and shifting the center of gravity while hitting 
were the most common problems in both male and female students. However, the issues like the timing of 
hitting, angle control of the wrist and racket, and the swing pattern seem more serious in the control group. 
Some students in control group only swing the racquet to touch the ball but ignored the importance of swing 
pattern and shifting the center of gravity (Figure 4).These incorrect forms also lead to a low score of shot- 
data, such as a negative value of ball spin level and the bad impact spot. 



► M 4 ) 0:^m9 _ 49 km h 94 


Figure 4. A Student Hit a Ball Without a Complete Swing Pattern 

Overall, those students in experimental group performed a relatively smooth swing pattern and had better 
shot-data scores, such as higher swing speed and spin level, compared to those in control group. We think the 
positive effect on the form and shot-data in the experimental group might be attributed to the reflection 
activities engaged them to view their own videos and observed the flaws. By examining their videos with the 
instructor’s demonstrations help them to correct the forms more effectively. Additionally, when they feel 
confident in their action, they are able to use power and got higher scores on swing speed or spin level. 
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3.3 How Females and Males Reflected Differently on Their Skills? 

Six weeks of reflection content related to the forehand and backhand skills were collected in experimental 
group. With a total of 135 posts were analyzed by an experienced tennis teacher based on critical tennis 
techniques. We count the number of times that each key technique was mentioned in the posts by the 
different gender. The percentages of each technique mentioned by different gender are presented in Table 2. 

According to the results, we can find each technique had a similar distribution between males and 
females. Therefore, male students are more likely to refer to the other aspect (11.96%), such as their 
expectations and emotions, compared to female students (2.56%). This result might due to the difference of 
psychological tendency between males and females. Males usually showed greater interest in sport than 
females, thus they will express more personal feelings while reflecting. 

In addition, male students pay more attention to the swing path problems (27.17%); but for female 
students, the firming wrist and racquet angle problems (26.28%), as well as the hitting timing (25.64%) were 
their major flaws (26.28%). These results echo the problems found in video analysis. Male students 
performed better on the timing for making a shot, but they usually ignored the importance of swing pattern, 
and the misjudgment of flight trajectory caused female students missed the timing for hitting and forced to 
change their forms. It revealed that both male and female students could correctly reflect according to their 
flaws. 

Both genders hardly mentioned the flight trajectory problem (Male: 0%; Female: 0.64%) and whether 
they hitting the sweet spot (Male: 2.17%; Female: 0%). Self-reflection can help students to become more 
aware of their strengths and weakness, and we think the reflective activities could facilitate tennis beginners 
to correct their form regardless of the gender. However, despite the sensor provided a precise impact spot 
data for each shot in videos, students rarely mentioned it in their reflection. We suggest further study should 
provide students a framework for reflection and stress how to interpret the shot-data in their reflective 
activities 


Table 2. Percentages of Students’ Reflection on Key Tennis Technique 


Reflection types 

Gender 

Male (N=10) 

Female (N=15) 

1. Identifying the flight trajectory of an approaching ball 

0.00% 

0.64% 

2. Proper timing for making a shot 

20.65% 

25.64% 

3. Control of the angle of the wrist and racket when hitting the ball 

20.65% 

26.28% 

4. Whether hitting the ball on the sweet spot 

2.17% 

0.00% 

5. Whether the swing path is proper 

27.17% 

22.44% 

6. Maintaining balance and shifting the center of gravity of body 

17.39% 

22.44% 

7. Others 

11.96% 

2.56% 


4. CONCLUSION 

This study aims to investigate the effect and gender difference through the use of sensor and video data as 
well as the reflective activities in a beginner tennis class. The results of this study showed that the learning 
outcome of forehand skill will not be affected by gender or group factor. However, for backhand skill 
acquisition, using our instructional approach could help female beginners learned better compared to those 
using the traditional approach, besides, could also reduce the gender difference in beginners. In addition to 
this, by analyzing the videos, we found male students performed better on the timing for making a shot 
compared to female students. The major differences between genders on shot-data scores are the swing speed 
as well as the spin level. Such a difference could be attributed to the physical factors that males usually have 
greater physical power and faster movement speed than females, these advantages help them to move to the 
position according to the flight trajectory and performed a relatively complete swing pattern. Overall, those 
students in experimental group performed a relatively smooth swing pattern and had better shot-data scores 
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compared to those in control groups. It is obvious that to incorporate the sensor data along with designed 
reflective activities could help both male and female beginners had better learning performance, especially 
while learning the more difficult skills, such as backhand groundstrokes. Finally, both male and female 
students had mentioned the critical techniques in their reflections with a similar distribution, besides, male 
students are more likely to refer to their expectations and emotions but the female students almost not. Male 
students pay more attention to the swing path problems, and female students thought the firming wrist and 
racquet angle problems, as well as the hitting timing, were their major flaws. These results echo the problems 
found in video analysis and indicated that students are able to reflect according to their flaws. However, even 
the sensor provided the precise shot-data and impact spot information, students were usually ignored the 
techniques about identifying the flight trajectory as well as whether hitting the sweet spot. 

This study showed that sensor could serve as a useful learning tool as well as an objective evaluating 
source for teaching tennis beginners. Proper use of the sensor data could enhance the learning performance 
and reduce the gender difference in sports learning. Future studies should use the sensors to collect data from 
a large sample of tennis beginning players so that the differences between genders could be thoroughly 
examined and analyzed. In addition, how to improve the instructional approach while using sensors in class 
to better fit both male and female students’ needs is another critical issue in this field. 
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ABSTRACT 

Transitioning to university is recognised as a challenging endeavour for commencing students. For commencing 
Computer Science students specifically, evidence suggests a link between poor performance in introductory technical 
courses, such as programming, and high attrition rates. Building resilience in students, particularly at the start of their 
academic journey, can potentially reduce the likelihood of student attrition. One way to develop resilience is through 
strong disciplinary engagement. We aim to increase student engagement by using popular low cost embedded platforms 
that provide a physical environment for novice programmers. This physical programming environment allows novice 
programmers to better appreciate the broad application of programming to everyday objects. Several experiments with 
the physical computing devices were designed and implemented for a first-year Computer Science programming course. 
These experiments were evaluated through a survey. Results indicate that whilst students found the environment more 
challenging than the normal computer-based environment, they felt more engaged in the programming process and 
enjoyed seeing the practical applications of hardware programming. 


KEYWORDS 

Programming Concepts, Computer Science, Physical Computing, Arduino 


1. INTRODUCTION 

Transitioning to university is a challenging endeavour [Baik, Naylor and Arkoudis, 2015]. Lizzio’s Five 
Senses of Success model [Lizzio 2006; Lizzio and Wilson 2010] outlines a number of aspects that can 
contribute to a successful transition. One of these indicators is the level of disciplinary engagement. 
Improving disciplinary engagement can build resilience in students and potentially reduce the likelihood of 
student attrition. Building resilience is particularly important at the start of one’s academic studies. 

There is ongoing evidence that commencing Computer Science students struggle with technical courses 
such as programming, and that poor performance in such courses is linked with high attrition rates 
[Watson and Li, 2014]. We posit that part of the challenge for students in these technical introductory 
programming courses is low disciplinary engagement. Some factors involved in this could be due to the 
abstract and inscrutable nature of the programming environment. A student cannot see inside the computer to 
understand how their program is executing and there are limited ways to link the behaviour of the program 
with its output [Milne and Rowe, 2002]. This becomes somewhat less problematic as the student’s 
experience grows, but it can be a significant hurdle and source of frustration for the novice programmer and 
can lead to diminished engagement. 

There is growing evidence [Richard 2010, Rubio et al. 2014, Maia et al. 2009] that programming in a 
physical environment using some form of physical computing modules is beneficial to the novice 
programmer. There is a direct connection between the program as written and the visible behaviour of the 
physical devices. The behaviour of the device (e.g. very high pitch tone or the motor moved too far to the 
left) can provide better, visual and auditory feedback to the novice programmer on the correctness of their 
program. This can alleviate some of the frustrations experienced by novice programmers. 

The approach taken in this paper is to focus on improving disciplinary engagement, in this case 
programming in Computer Science. We aim to increase student engagement by using popular low cost 
embedded platforms that are the backbone of the “Internet of Things”. The use of embedded platforms helps 
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student appreciate the broad application of programming to everyday objects and gives them a concrete focus 
for practical work that they can take home and integrate into their own study environments. Students will 
solve real world problems of a physical nature using the same technology used in everyday “smart” 
appliances. Students will be able to physically observe the outcomes of their solutions to problems, rather 
than as a text printout on a screen. They will also be able to reflect on the effectiveness of their solutions and 
will be able to experiment to obtain alternate solutions and evaluate the effectiveness of these in comparison 
to other potential solutions. 

We have designed several experiments with the Arduino physical computing devices [Arduino] for a 
first-year Computer Science programming course. Despite students finding the environment more 
challenging than the normal computer-based environment, student feedback (via a survey) strongly indicates 
that students felt more engaged and appreciated the hands-on practical experience of hardware programming. 

The remainder of this paper is organised as follows. In Section 2, we describe the programming 
experiments that we designed on the Arduino platform. We present our survey methodology and discuss the 
student feedback on the experiments in Section 3, and offer our conclusions in Section 4. 


2. PROGRAMMING EXPERIMENTS ON THE ARDUINO PLATFORM 

Arduino is an open-source physical computing platform that has been designed to make learning electronics 
and programming easier for students and beginners. It consists of a software integrated development 
environment (IDE) that allows users to write programs in the C/C++ programming languages, which are 
compiled, uploaded and executed on a microcontroller hardware platform (Figure 1). 

In our first-year Computer Science introductory programming course, we have designed several 
laboratory modules with the Arduino platform to enhance the students’ understanding of the basic 
programming concepts, and to enable them to see the practical real-world applications of the C/C++ 
programming languages. 



Figure 1. The Arduino Integrated Development Environment and the Arduino Hardware Platform 
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2.1 Utilising a Digital Compass to Control the Servo Motor’s Rotation 

This laboratory module consists of two sub-experiments (Figure 2): 

In the first sub-experiment, students are directed to connect the servo motor to the Arduino board, and 
provided with a simple program that demonstrates how the servo motor can be set to point to a specified 
position (or angle). Students are then asked to extend the simple program to make the servo motor point to a 
series of different angles, using an array. This experiment teaches students the programming applications of 
arrays of numbers as well as some simple arithmetic operations. 

In the second sub-experiment, students are asked to connect the digital compass to the Arduino board and 
provided with a simple program that shows how the Arduino board reads the data of the heading direction 
computed by the digital compass. Students are asked to convert the heading direction reading from radians to 
degrees, and then feed the reading into the servo motor via the Arduino board. Therefore, the servo motor 
will point to a specific angle corresponding to the digital compass’ reading. Students are also asked to group 
the program code corresponding to the servo motor and digital compass into separate functions, thereby 
giving them the opportunity to practise calling and returning values from functions. 

2.2 Generating Melodies with Sound Buzzer 

In this laboratory module, students are given a sound buzzer connected to the Arduino board (Figure 3), and a 
program that demonstrates how the sound buzzer can be used to produce tones at particular frequencies. 
Students are then asked to extend the given program to play a melody (tune), with the students also given a 
list of notes and their corresponding frequencies. In writing the program to play a melody, students get to 
practise and apply their knowledge of the basic programming constructs such as selections, loops, and 
functions. 



Figure 2. Servo Motor & Digital Compass 
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Figure 3. Sound Buzzer 



Figure 4. Digital Multi-Function Shield 


2.3 Click Counter with Digital Multi-Function Shield 

In this laboratory module, students are given a digital multi-function shield attached to the Arduino board 
(Figure 4). The multi-function shield has both a seven-segment display and several buttons. Two simple 
programs are given to the students, demonstrating how to detect button presses and how to drive the 
seven-segment display. Students are then asked to write a program that implements a counter, with the button 
presses as input. Essentially, the seven-segment display will show the value of the counter, with the value 
incrementing from 0 to 100 for each button press, and then looping back to 0 again. This exercise gives the 
students opportunities to write a program that utilises functions, the selection construct, arrays, and the 
modulo/division arithmetic operations. 
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3. METHODOLOGY 

In our first-year Computer Science introductory programming course, students attend weekly two-hour 
lectures and are given weekly homework programming exercises. The Arduino laboratory experiments 
described in Section 2 were offered to the students during the weekly two-hour laboratories (after having 
been introduced to the basic programming constructs), and are designed to enhance the students’ learning 
experience and engagement with the course. These Arduino labs were optional and were not assessed. During 
the lab sessions, both the lecturer and tutor would provide one-on-one assistance to the students, giving hints 
and tips on how to solve the programming problems. 

In this study, our objective is to determine the effectiveness of the Arduino experiments on the students’ 
understanding and application of the basic programming concepts, and hence their engagement with the 
course. We developed a set of survey questions, which were distributed to the students during the laboratory 
sessions in the last week of the semester. The survey questions asked the students to grade their experience 
with the Ar duino experiments on a 4-point scale from Strongly Disagree (1) to Strongly Agree (4). Student 
responses were completely anonymous. For the survey, the cohort size is 70 with n = 27 students 
participating (38.6% response rate). It should be noted that since the survey was conducted in the last week of 
semester in laboratory sessions with no summative assessment, student attendance was well below average 
for that week’s sessions. 


4. RESULTS AND DISCUSSION 

Student feedback has been encouraging, as seen from their responses to the survey questions. The questions 
are listed in Table 1, while the student responses are tabulated and shown in Figure 5. Some student 
comments are also shown in Table 2. These student comments are specifically related to the Arduino 
experiments, and were obtained from a separate end-of-semester course evaluation survey that is run by the 
university. 

Figure 5 shows that while the students found the programming exercises on the Arduino platform to be 
more challenging compared to programming on the normal computer-based environment (Q1 and Q2), they 
appreciated the practical aspects of the Arduino programming exercises and were more engaged in the 
process (Q3, Q4, and Q5). Students found the Arduino programming exercises to be more interesting and 
easier to troubleshoot as they were able to easily see (or hear) the “outputs” from their programs. For 
example, during the “Generating Melodies with Sound Buzzer” experiment, students enjoyed troubleshooting 
their code to generate an actual melody from the initial screeching tone that their programs generated. 
Student comments indicated that they had fun doing the Arduino experiments, and they were interested to 
further explore the capabilities of programming the Ar duino hardware. 


Table 1. Survey Questions on the Arduino Experiments 


Questions 

Description 

Ql 

The Ar duino programming was easier than the normal programming 

Q2 

I was more confident in Ar duino programming than in normal programming 

Q3 

The Ar duino programming helped me to explore the practical uses of C programming in real 
world applications 

Q4 

The Ar duino labs were interesting 

Q5 

I want to learn more about Arduino programming 
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Table 2. Student Comments on the Arduino Experiments 


Student 

Comments 

1 

I appreciate the whole idea to get a hands on approach to programming hardware 

2 

Engaging and interesting content, which I am excited to explore further 

3 

Arduino tutorials were fun 

4 

I enjoyed the Arduino 



□ Strongly Disagree 
E2 Disagree 

□ Agree 

■ Strongly Agree 


Figure 5. Student Responses to Arduino Experiments 


5. CONCLUSIONS 

We have designed several programming experiments with the Arduino physical computing devices and 
offered them to students in a first-year Computer Science introductory programming course. The aims are to 
enhance the students’ understanding of the basic programming concepts, and to allow them to see the 
practical real-world applications of C/C++ programming languages. Student feedback indicates that while 
they found the programming exercises to be challenging, they were more engaged in the process and were 
appreciative of the practical experience of hardware programming. This bodes well for the disciplinary 
engagement process since programming is a key threshold concept area that is foundational to all of 
Computer Science. 

For our future work, we plan to provide more preparatory lecture materials on the Arduino platform to 
scaffold the students’ initiation to the Arduino programming experiments. We will also design more Arduino 
experiments to give the students opportunities to write programs using more advanced programming 
constructs like linked lists and pointers. 
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ABSTRACT 

Undergraduate students entering university arrive with certain expectations as to how their degree program will equip 
them to enter the world of work. Students are aware of the competitive nature of the modern day labor market and, as 
seen in this study, there is a majority belief that their program of choice and the good reputation of the university will 
give them an advantage in increasingly competitive labor markets. This competition and the relentless trend of 
automation are placing downward pressure on the numbers of otherwise employable graduates. In this study, we use an 
established research instrument (Rothwell et al, 2008) to examine the expectations and self-perceptions of undergraduate 
students, considering dimensions of self-perceived employability in a cohort of 300 Information Technology (IT) 
students (136 responses) at a multi-campus, metropolitan university. We then contrast these results with the perceptions 
held by university students across multiple domains and professions. The results indicate that the IT students perceive 
themselves to be more employable relative to other professions. 
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1. INTRODUCTION 

It is no overstatement that labor markets in the early decades of the 21 st Century are experiencing 
unprecedented disruption. Not since the Industrial Revolution have such pressures been so glaringly evident. 
Contributing factors include the technological trend towards the automation of process-driven jobs, the 
outsourcing of labor to low-cost countries and the demand for more family-friendly, flexible employment 
arrangements (Stanwick, Lu, Rittie, & Cicelli, 2014). Last years’ graduate labor market is reportedly the 
toughest on record. There are more people completing degrees with fewer jobs, resulting in lower rates of full 
time employment (Norton & Carroll, 2015). 

The high cost of university/college education has traditionally been justified on the basis that graduates 
derive significant financial and personal benefits from higher education (Tomlinson, 2008). But the evidence 
tells a different story; it would appear that a university degree is no longer a guarantee or even an assurance 
for the securing of a job, as it once was. Recent findings suggest that it is taking graduates an average of 4.7 
years to find full-time employment after graduating (Healey & Lewis, 2016; Stanwick et al., 2014; Walsh, 
2016). It should be noted however that this 4.7-year figure includes all young people (aged 15-24 years), 
irrespective of whether they went to university, did a vocational education training course, or finished their 
education at high school (Healey & Lewis, 2016). 

What is perhaps more telling is that 30% of young people aged 15-24 are unemployed, while the 
proportion of young people in full time work has decreased from 52% in 2008 to 42% in 2014 and. at the 
same time, the proportion of people in part time and casual work has increased. Only 65% of university 
graduates will be in full time work four months after graduating, down from 84% in 2008. In the face of these 
trends, it is understandable that young people are staying in education longer (up from 72% in 2008 to 78% 
in 2014 among 15-19 year olds), on the expectation that acquiring specialized skills at postgraduate level will 
potentially help to prepare them for a future in which specialized skills are a necessity (Stanwick et al., 
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2014). The economic downturn and high unemployment rate may have led undergraduate students to an 
alternate course of action; to pursue practical courses that will potentially increase their employability in 
fields with recognized opportunity (Wu, 2011). In China, for example, students are choosing 
vocationally-oriented programs (Zhou & Lin, 2009), such as business and finance, sciences, engineering and 
architecture, rather than the humanities and social sciences. This trend may indicate that students consider the 
usefulness of a degree in gaining employment as a critical factor in choosing undergraduate programs (Lai, 
To, Lung, & Lai, 2011). Notwithstanding congestion in the labor market for some in-demand professions, 
there are still too many who have undertaken degree programs with limited jobs available (Zhou & Lin, 
2009). As a result, 11.6% of bachelor degree graduates available for full-time employment in 2014 were still 
looking for employment, 20.3% were working on a part time or on a casual basis while continuing to seek 
full time employment and 20.8% went on to further study (Guthrie, 2015). 


2. DIMENSIONS OF EMPLOYABILITY 

2.1 Defining Employability 

Theorists suggest that “employability is based on the competition for credentials as employers use them to 
screen out unsuitable applicants” (Brown. Hesketh, & Wiliams, 2003, p. 116). A more nuanced view is that 
employability is a subjective discourse that describes the way people perceive and understand the labor 
market, as well as their dispositions and attitudes towards their future in that labor market (Tomlinson, 2007). 
Hillage and Pollard (1998) define employability as having the capability to gain initial employment, maintain 
that employment and subsequently obtain new employment if required. A limitation of this last definition is 
that it is the labor market, rather than the capabilities of individuals seeking a place in it, that determines 
employability. Employability therefore varies according to economic conditions. When jobs are in short 
supply, graduates become less employable because there is a plentiful supply of experienced workers. From 
this we might deduce that we should not try to define employability solely in terms of individual factors. 
Brown et al (2003) thus propose that employability is a relative concept that is a function of supply and 
demand in the job market. Employability depends on two factors; how well one fulfills the requirements of a 
specific job and one’s relative standing in a hierarchy of job seekers. If there were more jobs available then 
applicants, then all applicants with the right qualifications and skills would be employed. However, even 
when the economy is buoyant, this is far from reality. 

We shall use a definition that takes account of the self-perceived employability of bachelor students: 
employability is “the perceived ability to attain sustainable employment appropriate to one’s qualification 
leveF (Rothwell, Herbert, & Rothwell, 2008, p. 2). 

2.2 Employability Factors 

2.2.1 University Reputation 

The reputation of one’s alma mater is a valuable commodity for students seeking employment. There is 
ample evidence to suggest that employers respond positively to institutional rankings and that degree holders 
from universities with good reputations have better chances of being employed. A study at the University of 
Sussex found that employers place heavy reliance on institutional reputation gained via rankings in the Times 
Higher Education supplement. One in four graduate recruiters cited university league tables as their main 
source of information about qualifications and standards. Moreover, employers tend to rely on league tables 
as a method for pre-selecting candidates, targeting graduates from the same 10-20 high reputation 
universities (University of Sussex School of Education. 2006). Cranmer (2006) likewise argued that 
employer perceptions about the quality of graduates from certain universities and departments continue to 
influence transitions into employment. Recent research in Australia, however, suggests this may not be the 
case. The Australian Graduate Survey (AGS) (Graduate Careers Australia, 2014) reports on employment and 
salary four months following program completion. The AGS showed that, after controlling for gender and 
course studied, the type of university attended did not significantly affect graduates’ probability of having a 
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job (Norton & Carroll, 2015). The HILDA survey (2014) looks at longer term outcomes and likewise 
demonstrated that the type of university attended made little difference to whether or not a person had a job. 

Previous research shows that students themselves recognize the importance of putting the reputation of 
their institution to good use. According to Tomlinson (2008) students’ perceived that they had to do all they 
could to gain positional advantage and they thus attached considerable importance to grades, the profile of 
the institution and the additional human capital of postgraduate credentials. It was clear that the students in 
this study were concerned to capitalize upon the profile of their institution and status of the university as a 
means to gain positional advantage and/or to re-invest in additional study. 

2.2.2 Supply vs Demand 

Each year, many potential workers with similar degrees and practical experience enter the market and 
compete for a small number of positions (Roulin & Bangerter, 2013) and there are indications that the growth 
in the supply of potential graduate labor is not matched by corresponding rise in actual demand (Wilton, 
2011). As Birrell and Healy (2013) note, approximately 250,000 young people in Australia leave school and 
try to enter the workforce each year. Insufficient labor market demand is resulting in high rates of 
unemployment amongst those aged 15-24 (14.5% for 15-19 year olds and 9.5% for 20-24 year olds). Wu 
(2011) similarly notes that both the economic recession and expansion of higher education has resulted in an 
excessive number of college graduates, which has, in turn, led to a high graduate unemployment rate and a 
competitive labor market. Recent graduates from both developed and developing countries have experienced 
considerable difficulty finding a job and/or they settle for jobs that are not commensurate with their training. 
This has resulted in a range of problems associated with over-education and crowding out. Brown et al 
(2003) likewise claims that mass higher education is creating an over-abundance of potential knowledge 
workers. There is evidence of significant labor market congestion, leading some, perhaps many graduates to 
settle for lesser jobs for which they may be over-qualified. Research by Tomlinson (2007) revealed that 
students are themselves aware that the graduate labor market is highly competitive and that there are more 
graduates than jobs. In addition, students perceive the limitations of their hard currencies and that an 
inflationary rise in formal credentials has lowered their value and currency. They perceive that their degree 
alone is not enough and does not represent a badge of sufficient distinction in their pursuit of graduate jobs. 

The problems created by an over-supply of young graduates are compounded by the fact of older people 
remaining in employment when they might otherwise have retired. The percentage of those aged 60-64 
participating in the workforce has also increased (Walsh, 2016). As a consequence, young Australians face 
competition from both global labor market pressures and an ageing local population. Research by Australian 
National University demographers (McDonald & Temple, 2010) supports this assertion. They found that 
even with zero net migration, the numbers exiting the workforce between the ages of 55 and 64 do not exceed 
those entering in the 15-24 year age group. Employment competition is thus most evident amongst young 
people, who have to face a situation in which an increasing percentage of older persons are staying in work, 
resulting in fewer vacancies through the exits of older workers a decade ago (Birrell & Healy, 2013). 

2.2.3 Skills and Abilities 

Access to higher education is seen by many as an equalizer of opportunities for people across the social 
spectrum. Individual attainment of marketable skills and formally recognized qualifications enables 
individuals to overcome social disadvantage (Wilton. 2011). However, recent research suggests that 
employers are attaching less importance to academic credentials (Brown, Hasketh, & Williams, 2004) and 
that there may be a mismatch between graduates’ level of qualification and its market utility (Tomlinson, 
2008). Survey data collected from employers shows dissatisfaction with young people’s level of business and 
customer awareness, self-management skills, problem solving abilities, literacy and numeracy skills. In 
addition, the baby boomer generation is living and working longer and they bring experience and skills that 
young people may not have (Walsh, 2016). Brown et al (2003) suggest that employers often report that 
university graduates lack business awareness and are poorly prepared for the work environment, while 
Cranmer (2006) similarly notes that employers have expressed concerns that undergraduate programs are 
failing to provide graduates with the necessary skills for their careers. 

The perceived deficits in graduate skills puts the onus on employers to complete their young new-hire’s 
education with the knowledge, skills and abilities seen to be important (Fugate, Kinicki, & Ashforth, 2004). 
To be competitive, graduates must add value to their primary academic credentials in order to distinguish 
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themselves from the many others with equivalent degrees in a competitive labor market (Cranmer, 2006). 
Tomlinson (2008) explored the way in which higher education students view the role of their degree 
credentials in shaping their future employment prospects. He found that, while students’ continue to ascribe 
importance to their degree, they also perceive that the role of their academic credentials in securing 
employment is declining. While some students maintained an idealized view of their employability, most 
students anticipated a more difficult process of career progression. They viewed the labor market as 
increasingly flexible and high risk and were aware that they would not simply walk into jobs but would, 
rather, have to manage their labor market experience and profiles to realize their goals. They took an active 
approach to managing their future employability and were aware of the need to optimize their credentials and 
be proactive in the management of their employability (Tomlinson, 2007). A study by Fugate et al (2004) 
similarly revealed that the majority of students surveyed did not believe that their degree would guarantee 
their future employment and that they needed to be strategic in order to distinguish themselves from their 
competitors. Approximately 25% of students in this study suggested that participation in extra-curricular 
activities could be used to gain positional advantage in a competitive labor market. In contrast, Wilton (2011) 
conducted a study with business and marketing students who were asked to assess the extent to which their 
undergraduate degree had contributed to the development of a range of employability skills. The graduates in 
this study reported that their programs of study contributed to the development of a broad range of 
employability skills that they believed would be valuable in a wide range of jobs and labor market contexts. 

2.2.4 Contrasting Results 

Given the international trend towards jobs becoming more difficult for university graduates to secure, the 
results of this project presents some anomalous findings. In this study of IT students, the generally buoyant 
attitude towards employability would appear to be at variance with the literature, which suggests that students 
generally perceive their situation rather more pessimistically. 

The variance is partially explainable by the dynamic and pervasive nature of the IT sector as it develops 
into a global industry that reaches into practically all aspects of modern life and which offers job prospects 
anywhere in the world without the restriction of needing to have a licenses to practice granted by a governing 
professional body. Another factor might be the emphasis that this particular university places on 
‘employability’ being built into its programs, a message which it communicates in its marketing. 

2.3 Method 

Rothwell et al (2008) constructed a research instrument that examines the expectations and self-perceptions 
of undergraduate students. This instrument has been well-validated by other researchers. Having gained 
Rothwell’s blessing, the authors of the current study used the survey-building tool Qualtrics to adapt this 
self-perceived employability questionnaire. Thus the study was conducted with a robustly tested survey 
instrument. The data and conclusions derived from the instrument would thus prove useful to contrast with 
the results of the current study. 

The survey consists of 20-items—four establishing demographic questions, including gender, age, highest 
completed education level and primary employment, and 16 Likert scale items presented on a 
standard 7-point scale (strongly agree, agree, somewhat agree, neither agree nor disagree, somewhat disagree, 
disagree and strongly disagree). 

Clearance was sought and obtained from the university’s research ethics committee. The link to the 
survey with supporting information was subsequently emailed to a cohort of 330 first year Bachelor of IT 
students at the very beginning of their university studies, i.e. two weeks after beginning their university 
studies. The test site is a large metropolitan university in Australia (40,000+ students across five campuses). 
The students for this study were drawn from three of the five campuses. These are located in 
demographically diverse parts of the extended metropolitan area. Around half of the students were 
Australians drawn largely from areas within a 50 kilometer radius of the three campuses. These areas were 
characterized by being in three distinct demographic categories; one low Socio-Economic Status (SES), one 
low to middle, one middle to high SES. The other half was international students, drawn from East Asia. 

Participation in the survey was voluntary and anonymous. No additional course credit or other incentive 
was offered for participation. A total of 136 responses were received, although Likert scale items received 
between 91 and 93 responses, suggesting that many participants discontinued with the survey after entering 
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demographic information. The survey was open for a total of two weeks, at the end of which time a report of 
survey responses was generated using the Qualtrics software. For Likert scale items (survey questions 16 and 
17), responses of Strongly Agree and Agree and Strongly Disagree and Disagree were conflated and 
converted into percentages, while responses to somewhat agree, somewhat disagree and neither agree not 
disagree were calculated individually as percentages. 

2.4 Results 

2.4.1 Demographic Variables 

Demographic information drawn from the survey includes age, gender, highest completed education and 
employment status, if any. 

Of the respondents, 89% (n=121) identified as male and 11% (n=15) as female. 70% (n=95) were aged 
between 17 and 19 at the time, while 21% (n=29) were aged between 20 and 29. 

74% (n= 101) indicated that Year 12 was their highest completed education level, while 26% (n=35) had 
completed a certificate level qualification. In addition, 62% (n=84) of participants identified as students, in 
response to the question: which best describes your primary level of employment? 

2.4.2 Likert Scale Items 

The participants were asked: “To what extent do you agree or disagree with the following statements?” 
Responses for strongly agree, agree and for strongly disagree and disagree have been conflated. 


Table 1. Survey Response Breakdown (Consolidated Likert Scale) 



Question 

Results 

1 . 

I achieve high grades in relation to my study 
(92 responses) 

50% (n=46) agreed, 

38% (n=35) somewhat agreed. 

2. 

I regard my academic work as top priority (94 
responses) 

71 % (n=67) agreed, 

18% (n=17) somewhat agreed. 

3. 

Employers are eager to employ graduates from my 
university (92 responses) 

47% (n=43) agreed, 

32% (n=29) neither agreed nor disagreed. 

4. 

The status of this university is a significant asset to 
me in job seeking (93 responses) 

53% (n=49) agreed, 

23% (n=21) somewhat agreed 

20% (n=19) neither agreed nor disagreed. 

5. 

Employers specifically target this university in 
order to recruit individuals from my subject area(s) 
(93 responses) 

26% (n=24) agreed, 

24% (n=23) somewhat agreed 

42% (n=39) neither agreed nor disagreed. 

6. 

My university has an outstanding reputation in my 
field(s) of study (93 responses) 

54% (n=50) agreed, 

24% (n=22) neither agreed nor disagreed. 

7. 

A lot more people apply for my degree than there 
are places available (93 responses) 

18% (n=17) agreed, 

42% (n=39) neither agreed nor disagreed, 
11% (n=10) somewhat disagreed 

10% (n=9) disagreed. 

8. 

My chosen subject(s) rank(s) high in terms of 
social status (92 responses) 

25% (n=23) agreed, 

29% (n=27) neither agreed nor disagreed, 
12% (n=l I) somewhat disagreed. 

9. 

People in the career I am aiming for are in high 
demand in the external labor market (92 responses) 

45% (n=41) agreed 

30% (n=28) somewhat agreed. 

10. 

My degree is seen as leading to a specific career 
that is generally perceived as highly desirable (92 
responses) 

63% (n=58) agreed 

17% (n=10) somewhat agreed. 
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11. 

There is generally a strong demand for graduates at 
the present time (91 responses) 

49% (n=45) agreed 

21% (n=19) somewhat agreed. 

12. 

There are plenty of job vacancies in the 
geographical area where I am looking (92 
responses) 

27% (n=25) agreed, 

34% (n=31) neither agreed nor disagreed, 

9% (n=8) somewhat disagreed. 

13. 

I can easily find out about opportunities in my 
chosen field (92 responses) 

49% (n=45) agreed, 

32% (n=29) somewhat agreed 

13% (n=12) neither agreed nor disagreed. 

14. 

The skills and abilities that I possess are what 
employers are looking for (91 responses) 

56% (n=51) agreed, 

27% (n=25) somewhat agreed 

12% (n=12) neither agreed nor disagreed. 

15. 

I am generally confident of success in job 
interviews and selection events (92 responses) 

50% (n=46) agreed 

26% (n=24) somewhat agreed. 

16. 

I feel I could get any job so long as my skills and 
experience are reasonably relevant (92 responses) 

64% (n=59) agreed 

17% (n=16) somewhat agreed. 


2.5 Discussion 

2.5.1 University Reputation 

In keeping with conventional wisdom, earlier research (Cranmer, 2006; University of Sussex School of 
Education, 2006) suggests that in Britain employers place significant reliance on institutional reputation to 
pre-select graduates. A graduate from Oxford or Cambridge would be regarded more favorably than one from 
universities at the bottom of the league table. For reasons that remain unclear this does not appear to be the 
case in Australia (Graduate Careers Australia, 2014; HILDA Survey, 2014; Norton & Carroll, 2015). This 
study has 47% of participants believing that the hiring managers want graduates from the same universities 
that they themselves went through. More than half of the students in this study (53%) nonetheless consider 
that the status of the university will be an asset to those seeking employment with 20% somewhat agreeing 
with this view. 

Paradoxically, while 54% of participants reported that the university they attend enjoys an outstanding 
reputation in their chosen field, only 26% believed that employers would give preference to their university 
when recruiting. While this cohort of undergraduate students perhaps hold an idealistic, if not naive, view 
about the likelihood of their finding full-time employment on completion, they nonetheless recognize that 
employers will choose them because of the institution they attend and that they will be competing with other 
equally, if-not-more, qualified graduates. 

2.5.2 Supply vs Demand 

Labor market statistics may be telling us that the supply of graduates in Australia exceeds demand (Birrell & 
Healy, 2013; Brown et al., 2003; Tomlinson, 2007; Wu, 2011), yet the results of this study indicate contrary 
perception. Only 18% of participants believe that more people apply for their degree than there are places 
available. 42% neither agreed nor disagreed with this question, telling us that the perceived demand for 
graduates in their chosen field exceeds the supply of graduates. This is emphasized by the participants’ belief 
that people in their chosen degree are in high demand in the external labor market, a view which 45% and 
30% agreed or somewhat agreed with. However, while participants generally believed that their degree was 
leading to a career that is highly desirable (63% agreed and 17% somewhat agreed), only 49% agreed that 
there is, generally, a strong demand for graduates at the present time and 21% somewhat agreed. We may 
conclude from this that while the respondents believe that their chosen degree is in demand and will likely 
result in their gaining employment, there may be graduates in other disciplines in which the supply of 
graduates exceed the number of positions available. 
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2.5.3 Skills and Abilities 

Employers appear to be attaching less importance to academic credentials than was previously the case 
(Brown et al., 2004); a view at variance with the perceptions of undergraduates, 56% and 27% of whom 
either agreed or somewhat agreed that they possess the skills employers are looking for. Moreover, 64% 
agreed and 17% somewhat agreed that they will find employment if their skills and experience are relevant, 
whereas prior research suggests that employers themselves have concerns about whether programs are 
producing graduates with the skills needed to for today’s workforce (Brown et al., 2003; Cranmer, 2006). 

2.6 Directions for Future Research 

An interesting future direction would be to make the current study into a longitudinal study conducted over 
three years. The longer study would examine the employability self-perceptions of (a) each new first year 
cohort, and (b) the progressive perceptions of the current first year cohort as they progress through second 
and third years. Such a study would help to assess whether or by how much original perceptions may have 
changed. 

In addition, it would be illuminating if in 2019 we compare the employment figures for the recently 
graduated IT students who are the subject of this current study, and to compare those figures with those of 
2018 and 2017. This would give some insight into whether the employability programs undertaken by the 
university are having a positive effect. 


3. CONCLUSION 

The results of the survey indicate a degree of optimism in the first year students that is understandable given 
the aspirations that led them to enroll in the IT program and the encouragement given by the university itself 
to believe that acquiring employability skills is a key component of their program. This is a successful 
outcome in the university’s efforts to generate a positive mind-set in relation to students’ future 
employability. 

Employment prospects in the IT industry appear to be more favorable than other less dynamically 
changing professions. It is an open question whether this optimism is warranted in view of the general 
evidence that suggests the supply of graduates on the labor market in many industries exceeds demand 
(Birrell & Healy, 2013; Brown et al., 2003; Tomlinson, 2007; Wu, 2011). 
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ABSTRACT 

STEM competitions are fairly widespread in middle schools and high schools, but do not commonly occur at the 
university level. We have developed a repeatable model for a one-day competition in which high school, community 
college and university students can build confidence in their own critical thinking abilities and develop enthusiasm for 
careers in science, technology, engineering and mathematics. The competitions are used to build confidence and 
excitement in students and to encourage them to consider choosing a STEM degree. 
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1. INTRODUCTION 

The United States is not producing enough STEM (science, technology, engineering and mathematics) 
graduates to fill available jobs. According to the Office of the Chief Economist of the United States, STEM 
occupations have grown at a rate of 24.4% in the last ten years while non-STEM occupations have grown 
4.0% [Noonan 2017]. STEM occupations are projected to continue growing in the future. Brian Kelly, editor 
of U.S. News & World Report states that “we need to focus our efforts on getting more kids, particularly 
women and African-Americans interested in pursuing STEM at a young age” [Laros 2016]. 

Fostering positive learning environments can increase student concentration and focus and contribute to 
student success in STEM. Self-efficacy ultimately determines if students overcome challenges that arise to 
persist in STEM, or resign [Hackett 1989]. Generally, if individuals participate in “manipulating, assembling, 
disassembling, constructing, modifying, and breaking and repairing components and devices,” their 
confidence increases [Baker 2007]. Providing hands-on interactive experiences for students in STEM can 
potentially increase self-efficacy. 

We have developed a STEM Challenge for high school, community college and university students in 
which students are given the opportunity to follow instructions, make decisions, and work with a team. The 
Challenge is an extra-curricular activity in which students are given several opportunities to find solutions to 
various problems without the pressure of being graded on their performance. In a fun and non-threatening 
atmosphere, students participate in problem-solving activities that help to develop critical thinking and 
technical skills. 

The format of the STEM Challenge has changed significantly since its inception with input from student 
participants, student mentors, industry advisors, and faculty advisors. In this paper, we will discuss each 
challenge, then we will lay out the logistics for setting up a STEM Challenge. Finally, we will discuss the 
outcomes that we have met to date. 
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2. PROGRESSION OF THE CHALLENGE 

As part of a computer science recruiting and retention grant from the state of Texas, our university, in 
partnership with a community college and representatives from local industries, instituted a Programming 
Challenge in 2008 as a pilot recruitment tool. Eleven teams with 34 contestants participated in the 
competition at two levels. Winners and participants won decent prizes including scholarships, internships and 
various gifts. Follow-up feedback indicated that 92% of the participants evaluated the Challenge as very good 
or excellent. This Challenge was successful in the intended mission: from the students who participated, 

7 enrolled in STEM programs within the first year [Davari 2007]. 

This idea was further developed and became part of a major STEM grant proposal submitted to NSF 
which was funded in 2013. The 5-year NSF STEP grant was for a project with several STEM related 
activities that included a yearly STEM Challenge as one of the activities, with the main goal of recruiting and 
retaining students in STEM fields [Davari 2016]. 

Winners of the challenge receive scholarships to the university ranging from $250 to $500 funded by the 
grant. In addition to scholarships, winners receive internships and prizes. A raffle draw provides other 
participants with prizes ranging from restaurant gift cards to solid-state drives. Each participant also receives 
a t-shirt. 

2.1 Robotics Challenge 2014 

The topic for the very first STEM Challenge was robotics. Faculty in the computing and mathematics 
divisions decided on this topic as an interesting and stimulating way to introduce computing to students. 

The challenge consisted of two levels of competition - beginner and advanced robotics programmers. 
Due to the cost of equipment, the challenge was limited to 15 teams, with a maximum of four members per 
team. Students who did not have any robotics experience were added to the beginner category. If any team 
member was a junior or above in college or they had any robotics experience, they were designated to the 
advanced category. 

Since this was a robotics challenge, students without any robotics experience needed assistance to 
understand how Arduinos and robotics operated. Two weeks before the competition a three hour, hands-on 
‘Tech Friday’ was presented to anyone who wanted to attend. This presentation was also videotaped and 
posted on the project website with access to all. 

During the challenge, each team spent 50 minutes at one robotics station, trying to complete up to 4 
different tasks, before rotating to the second and third station. Each task was assigned a point value, and tasks 
had to be completed in a particular order and demonstrated before receiving instructions for the next task. 
Each subsequent task became more challenging. 

The Morse Code station had students representing Morse code using the LEDs on the Arduino boards. 
The first challenge was to create a sequence of characters in Morse code using a LED. Each team was 
assigned to code the letters A, B, C, D, and E. For example, A is dot followed by dash, where the dot is 
represented by a l A second (or 250 millisecond) light and the dash is represented by a % second (or 750 
millisecond) light. The two are separated by a !4 second interval of no light. There should be a % second 
delay until the next letter. The next challenge in that station required the team to create the string “UHCL 
SJCD NSF” using Morse code and the LED. The final challenge in the station was to create a script that 
would turn a Bluetooth command into Morse code. The phrase was given at the time of judging. 

The Sensor station required students to work with different sensors and complete tasks. For example, one 
task was to set up a LED to begin blinking if a fire was detected, with the fire sensor calibrated to work for a 
bright light. Another challenge required the teams to read the temperature using the DHT11 temperature 
sensor and display it in Celsius on the LCD screen. 

The Tank station required the teams to move a tank, starting with just left, right, forward and backwards, 
and the final task having to get the tank to move through an obstacle course. 

Participants ranged from 7 th graders to college seniors. The registration closed very quickly, and many 
students and teams were turned away. Fifteen teams with 51 contestants participated in the competition with 

8 beginner teams and 7 advanced teams. 
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2.2 STEM Challenge in 2015 

At the external advisory board meeting for the NSF grant, several board members expressed concern that 
robotics only encompassed a very small portion of computing. They recommended that the challenge be 
more generic. Therefore, the theme of the next challenge was STEM as a whole. 

Over 8 different high schools and 6 different community colleges and universities were represented in the 
challenge. The challenge consisted of two levels of competition. The event was limited to 18 teams. Twelve 
beginner teams and 6 advanced teams participated in the event. 

Each team rotated through three stations: Game of Clues to challenge their math knowledge. Creative 
Inventors to showcase their imaginative skills and Robot Adventure to test their problem-solving and 
programming skills. 

The Game of Clues station showcased different ciphers with increasing degrees of difficulty. The first 
cipher was very simple, with A=l, B=2, ... , Y=25, Z=26. A phrase was given to the teams where the words 
in the phrase were in order, but, the letters were jumbled. The teams had to translate, unscramble and produce 
the phrase. The second cipher was more complex, where a coded message had to be translated using a 
formula incorporating modulus functionality. Another challenge used the Playfair Cipher. Yet another 
challenge was a Numbrix puzzle. 

The Creative Inventors station required the teams to create things. The first challenge required the team to 
build a free standing structure 3 levels tall, using only the notecards that were provided, while not talking to 
each other, using only one hand and not folding, bending, or tearing the cards. The second challenge built on 
top of the first one, where the structure now needed to be 4 stories tall, and also needed to hold the weight of 
a dinosaur that was given to them. Another puzzle was to figure out how four people crossed a bridge within 
an allocated time, with different constraints given. Another puzzle was, if 8 balls were given, with 7 of them 
the same weight, while one was slightly heavier, how to figure out the ball that is heavier by using a balance 
only two times. 

The Robot Adventure station required students to use the programming language Python to program a 
Turtlebot. The first challenge was to modify the code to create a turtle. The second challenge required the 
teams to move the turtle manually using the arrow keys. The next challenge was to teach the turtle to play 
golf. The turtle needed to move behind a ball in such a way that it was in-line with the ball and the hole and 
facing the ball. For the next challenge, the turtle needed to push the ball into the hole. For the final challenge 
the turtle graduated to a professional player and needed to play 9 holes of golf. 

2.3 STEM Challenge in 2016 

The previous year’s idea of making the challenge more generic was received well. We continued on this 
theme but expanded the challenge to accommodate more participants. Twenty-one teams (13 beginner teams 
and 8 advanced teams) registered for the event. 

In 2016, instead of a station comprising several small challenges, the Creative Inventors challenge 
comprised of one challenge. Teams had 35 minutes to use the provided materials to construct the tallest free 
standing structure within the allocated time. 

The Software Design challenge had different challenges for the two levels of competition. The beginner 
levels played an open-source game, where they had to make their way through the levels of the game by 
coding and gathering gems. The goal was to get through as many levels as possible in the allotted time. The 
advanced levels, on the other hand, had to play a text-based game named Python Warrior. They played as a 
warrior that is trying to find his way out of a tower. Each floor has a map. On each floor, the warrior’s 
objective was to find the staircase leading to the next floor. The game automatically awarded points 
depending on the actions. 

The Engineering Design challenge required participants to use the provided materials to construct the 
most effective parachute. The parachute needed to carry all of the provided marbles when dropped from the 
third floor. The parachute designed was required to slow the descent of the marbles as much as possible. 
Points were assigned according to the time the parachute took to land, with the longest time to descend 
earning the maximum amount of 40 points, and the next slowest receiving 3 points less, and so on. 
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Between each of these challenges, a 5-minute fun math challenge was added to break up the stress of the 
competition. One such challenge was to complete the Towers of Hanoi from the leftmost peg to the rightmost 
peg, where the team members stood behind a mark, and one team member moved ONE disk and returned to 
the back of the line, before the next team member got his/her turn. Another math challenge was a logic and 
deductive reasoning grid puzzle. The third math challenge was to decode an encoded message. 

2.4 STEM Challenge in 2017 

In 2017, we had the biggest turn out yet, with 29 teams (19 beginner teams and 10 advanced teams) and 111 
students. 

The stations comprised of a Math station. Engineering Design station and a Software Design station. This 
was the first challenge where all the challenges in a station were given to the teams, and the teams got to 
choose which challenges to attempt, and in which order. The challenges had different scoring values. At the 
end of the allocated time, the teams returned their packets. 

In the Math station, teams were given a packet with 11 challenges. One of the challenges was to solve a 
given puzzle to find which letter corresponded to which number. The puzzle was GTOM + PNAG = EG 
OAT, where G=5, A represents an odd digit and 6 and 4 are not used. Another puzzle was to figure out how 
many squares were there on a checkerboard, based on a picture of it. Another challenge was to figure out 
how many ways to make change for 50 cents, using nickels, dimes and quarters. Some of these were very 
easy, while some needed a lot of reasoning. This was done purposely to make sure that the teams felt a sense 
of accomplishment. 

In the Software Design station, the teams were provided with a Challenge List. It listed all the challenges, 
their IDs, and their score value. The teams chose which challenges to attempt and the order in which they 
attempted them. They were allowed to program in either C++, JAVA or Python. The challenges were 
auto-tested and their scores were updated with the points. All the team scores were displayed live on a large 
screen, so that they were aware of their status. On the screen, each team was represented by its own avatar. 
Some of the easy challenges were to find the average and the median. Some more complex challenges were 
to find the least common denominator, to find if a number is a prime number and to find the convert second 
to time. 

In the Engineering Design station, each team was provided with $500 worth of paper money and an egg. 
Their goal was to design a system that will carry an egg from the third floor to the ground floor without 
breaking it, while spending the least amount of money for materials. The team that designed a system that 
carried an egg from the third floor to the ground floor without breaking it in the allotted time, while spending 
the least amount of money for materials received full points with each consecutive team with the least 
amount of spending receiving 2 points less. 


3. EVALUATION 

A mixed methods approach was used by an external evaluator to collect both quantitative and qualitative data 
through the distribution of both pre- and post-surveys, and open-ended questions. Using a Likert scale, 
surveys gauged student interest in STEM (and more specifically technology) prior to and following 
immersion in each Challenge. As well, student participants of the STEM Challenges were requested to 
provide candid feedback in response to open-ended survey questions. The data collected by the external 
evaluator was analyzed and disseminated to UHCL administrators to improve future student STEM 
Challenges. The narrative that follows provides a brief synopsis of the most compelling data collected during 
these Challenges. 

Participants of the Robotics Challenge were queried about their natural inclination to learn technology. 
Fifteen students submitted both pre- and post-surveys. No qualitative data was collected. The pre-survey 
indicated that only 40% of students felt they possessed an “extremely high ability.” However, data yielded 
from the post-survey, immediately following the intervention, indicated that 67% felt their technological 
aptitude was “extremely high.” In addition, students were prompted to provide feedback relating to their 
interest in robotics both before and after the intervention. The pre-survey indicated 40% of students were 
“extremely interested” in robotics, while post-survey data revealed that 50% felt their participation in Tech 
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Friday increased their interest in robotics. While 10% is somewhat incremental, it does suggest an impactful 
intervention. 

Analogous to the Robotics Challenge, the STEM Challenges facilitated between 2015 and 2017 allowed 
for data collection from students before and immediately following participation. Exactly 74 students 
participated in the pre-survey, while 95 students submitted responses to the post-survey. A limitation to this 
data collection was the ability to distribute pre-surveys prior to the event (hence the variation in responses), 
due to participant arrival times and other logistics. Nonetheless, the data captured suggested a successful 
intervention. Prior to the STEM Challenges, nearly 35% of students suggested their technological ability was 
“extremely high.” Subsequent to the STEM Challenges, 44% of students responded that their technological 
capabilities were “extremely high,” a notable increase. In addition, the pre-survey requested that students 
gauge their knowledge relating to careers in STEM fields. A mere 14% responded that they were “extremely 
knowledgeable.” However, 25% more students suggested they were “extremely knowledgeable” about 
careers in STEM fields in response to the post-survey. As well, students confirmed their interest in STEM 
was strengthened as a result of their participation in this event. Approximately 43% of students inferred their 
interest in STEM was moderate, prior to the intervention. More than 55% of students responding to the post¬ 
survey “strongly agreed” that their interest was strengthened because of their participation in the STEM 
Challenge. 

Qualitative data was also collected from student participants at the conclusion of each STEM Challenge. 
Open-ended questions prompted students to offer feedback, and suggestions for future improvements. 
Initially, students were asked, “What did you like most about the STEM Challenge?” A sample of these 
responses is detailed below. 

• “I like the cooperation the Challenge requires to solve problems.” 

• “I learned about the skills I have, and what I need to improve.” 

• “Fun, challenging, helps you in a fun way to see what your strengths and weaknesses are.” 

Student participants were also requested to provide feedback on what they would change about the STEM 

Challenge. A sample of their responses are outlined below. 

• “Actual live science problems and questions, some anatomy, chemistry, physics.” 

• “I think it was great. With the programming, I think it would be helpful to see some feedback 
while programming.” 

• “Include more physics.” 

The final question of the post-survey urged students to offer any additional information or 
recommendations for program administrators. A sample of their responses are provided below. 

• “Please keep doing the STEM Challenge every year.” 

• “I had great fun today!” 

• “Good event, overall really fun.” 

Conclusively, the data collected between 2014-2017 infers that student self-efficacy was strengthened as 
a result of participation in these experiential events conducted by UHCL. Students with minimal exposure to 
technology received hands-on experience, networking opportunities, and support from their peers and 
faculty. As a result, STEM Challenges have become wildly popular not only at UHCL, but throughout the 
larger Houston academic community. 


4. OUTCOMES 

The event has progressively grown over the years. Through the four challenges, we had a total of 293 
students participate in the event. Twenty-three student mentors and research assistants participated in the 
organization and facilitation of the event over the years. Sixty-eight faculty, alumni, industry advisors 
participated in various capacities, such as judging and advising. 

Overall, student participants have been very happy to attend the challenge. Pre- and post-surveys are 
given and Table 1 shows some of the student comments on the open-ended survey questions on the 
post-survey. 
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Table 1. Open-ended Student Survey Answers 


It made me realize that I am good with technology (2014) 

I loved solving the logic and math based problems. The cryptologic questions were well made and 
challenging (2015) 

I would've liked to have a different language than Python to code (2015) 

I loved and really enjoyed the Python game from code combat, it was utterly the fastest way I have been 
introduced to a program language (2016) 

1 liked everything. The activity was fun and I learned a lot about robots (2014) 

The event was amazing and I really cannot name anything that was worth changing. (2016) 

1 learned about the skills 1 have to work to improve (2017) 

The challenge taught me to think outside of the box and learn to communicate my opinion without worry 
(2016) 

Fun, challenging, helps you in a fun way to see what some strengths and weaknesses are (2017) 

It was a lot of fun and the challenges had a great mix of problem solving and technical ability (2015) 

1 thought I wouldn’t be able to understand robotics, because 1 don’t have experience. The staff was really 
nice though and helped me and my team (2014) 

Very fun! Will come back again and win (2015) 

I think it was great. With the programming, I think it would be helpful to see some feedback while 
programming (2017) 


Throughout the four years, 24 internships were offered to participants of the STEM challenge. Nineteen 
students utilized this opportunity. When asked for feedback, one employer said “...extremely impressive in 
the way they understood, analyzed, designed and implemented a solution to the problem presented by the 
project. They were self-motivated and was creative in the way they found solutions. I would rate them as the 
best internship group we have ever worked with. If the STEM program can provide this type of quality 
interns to the industry, the program would have an excellent future.” 

Since 2016, the judges were also asked to provide their feedback about the event. Table 2 contains some 
of the comments from judges when asked what they thought about the STEM challenge. 

Table 2. Open-ended Judge Survey Results 

My students had nothing but praises for the event. They liked the idea of thinking outside the box (2016) 

My only concern is the programming as you used Python. For my students, it was not a problem. 
However, if some students had not been exposed to C++, Java and/or Python, they would be lost (2016) 
Make sure that students participate for the challenge, not the incentives (2016) 

1 think that most contestants had fun with the engineering challenge. I liked it because you were able to 
see how each team came up with different concepts (2017) 

All of my students left wanting to know how to program (2017) 

1 had three students participate last year, I had 20 interested this year. They are excited about it (2017) 


5. CONCLUSION 

Figure 1 shows the growth of the number of student participants in the challenges from 51 in 2014 to 111 in 
2017. The number of students in our computer science, computer information systems and information 
technology degrees has grown in the same time frame. While the growth of these 3 programs cannot be fully 
attributed to the success of the STEM challenges, we feel that the excitement of the events has impacted not 
only prospective students, but also current students, and even faculty. 
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TEAMS AND PARTICIPANTS OVER 
THE YEARS 


# of teams M # of participants 



Figure 1. Team and Participant Numbers at the Various Challenges 

The project team is very interested in increasing the number of women participants in the challenges and 
in our STEM degrees. Figure 2 shows that women participation grew to 22.5% in 2017. This is very exciting 
and encouraging to the project team. 



Figure 2. Women Participant Numbers at the Various Challenges 


The Challenges introduced students to both basic and advanced STEM concepts. Generated data 
conclusively infers that only moderate alterations should be considered for this program. As a result of 
engagement in this event, student participants acknowledged an increase in STEM interest. Additionally, 
despite possessing minimal experience in both programming and robotics, most student participants were 
successful in the completion of these events, and subsequently implied increases in confidence. 

However, the data derived also illuminates the need for precise and less complex instructions. Uncertainty 
relating to instructions could be explained by variations in age groups among student participants. Some 
participants also recommended, “pairing” middle school and high school students in groups with university 
students to make the challenge more equitable. 

Highly engaging activities like the Programming Challenge can be used and replicated as a model by 
institutions to ensure that self-efficacy among students, specifically in STEM fields, is achieved. 
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ABSTRACT 

It is understood that, for mentally disabled people, it is hard to generalize skills and concepts from one setting to another. 
One approach to teach generalization is solving the problems related to their daily lives, which helps them to reinforce 
some of their behaviors that would occur in the natural environment. The aim of this study is to develop a gesture based 
game to teach basic life skills to mentally disabled kids by a motion sensing device. To this end, a vacuum cleaning 
digital video game was designed by using the Unity3D game engine, and Microsoft Kinect, based on spiral development 
methodology. Tests were conducted in two special education schools with the help of a special education expert. The 
game prototypes were tested on various spectrum of mentally disabled children, and a final game was designed after 
several iterations. Results were extracted by observation of the participant’s performances. Results have shown that this 
system positively helps children’s basic life skills learning. 


KEYWORDS 

Special Education, Game, Motion Detection, Basic Life Skills 


1. INTRODUCTION 

Nowadays, usage of the digital video games in education is common in the world. Papastergiou (2009) and 
Griffiths (2002) confirm the positive effects of the digital video games in education. 

In his study, Inal (2011) presented a set of design principles which can be used as a guideline to develop 
body interactive educational games for children. Four physical interactive video games were designed based 
on these principles. The results of the study inform that usage of big camera screen increases the children’s 
motivation during gameplay. 

An Augmented Reality based motion-sensing software, could enhance the learning desire on pupils, and it 
may positively affect their learning procedure (Cai, Chiang, Sun, Lin, & Lee, 2017). In one of the earlier 
studies about the Kinect and its potential in education, Hsu (2011) found that it increases children’s creativity 
and classroom interaction. She argues that the Microsoft Kinect, in comparison with interactive whiteboards, 
is more useful. 

Recently, researchers have started to pay attention to use Microsoft Kinect in special education. Teaching 
methods in special education usually get along with the additional specific services such as rehabilitation, 
speech therapies, and so forth, to teach the disable children efficiently. Unlike the general education, in 
special education, people need individual help with considering their level of disabilities, and educational 
methods should be considered individually for them. Video games may have a positive effect on children 
with special needs and may be used continuously in special education (Ruggiero, 2013). In their research 
Zafmlla, Brashear, Starner, Hamilton, and Presti (2011), assessed the effects of the Microsoft Kinect to teach 
sign language to deaf people. They used to teach the sign language to deaf people by a CopyCat system. This 
system consists of a computer game which is designed specifically to teach the sign language along with a 
wearable controller and a camera to tracking the eye and hand movements. Although the performance of 
using wearable controllers is better than Kinect, from many other aspects Kinect is preferable. In their 
research, Lange et al. (2012) proposed a physical interactive game ("JewelMine") to improve motor skills of 
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disable people by using Microsoft Kinect. There are some objects on the screen which are placed around the 
players’ current positions. A player should retrieve these objects one by one based on their orders. Results 
confirmed the effectiveness of the game as a rehabilitation tool. In addition, a study by Camara et al. (2017) 
confirms that Kinect could be an effective device to rehabilitate the motor skills in teenagers with Cerebral 
Palsy. Boutsika (2014) recommends the usage of Kinect as an assistive device for children with autism along 
with "Mnemonic Techniques". The researcher focuses on the existing games such as "Kinect Adventures" 
and their potential application in special education. Chang, Chou, Wang, and Chen (2013) offered a system 
called Kinempt. It is designed to help the individuals with cognitive impairments to involve in food 
preparation training by using Kinect gesture recognition service. The study results indicated that Kinempt 
and image recognition technology may facilitate to learn job skills by children with cognitive disabilities. 

This study overview shows that interactive body movement games have many applications in various 
areas and may be helpful in both traditional education and special education. There are various types of 
special students. Mentally disabled people with a various spectrum of disability are the most vulnerable 
group of the special students. Students with mental disabilities often have problem in generalizing 
information from one setting to another. An alternative way to teach generalization is to teach students to 
solve problems pertinent to their daily lives and to reinforce behaviors that would occur in the natural 
environment. In this study, we propose a system to augment the learning environment for mentally disabled 
students in order to easily cope with life skills by game and fun. The game was developed for motion sensing 
input devices by following a spiral iterative methodology. Therefore, we postulate that it is necessary to 
design body movement games for children with mental disabilities to teach the basic life skills. 

In the present paper, a digital vacuum cleaner game was designed by using the Unity3D game engine and 
Microsoft Kinect. Before the project started, a permission had been taken from the Human Subjects Ethics 
Committee. The game was based on the designed scenario to teach vacuuming skills to mentally disabled 
people precisely by moving their hands and bodies like a real vacuuming act. As a result of this study, the 
effects of this game on the mentally disabled children performance were assessed. 


2. GAME DEVELOPMENT PROCESS 

A digital vacuum cleaning game was developed by using the Unity3D game engine and Microsoft Kinect 
motion sensing input device by following a spiral iterative methodology. At the beginning, game concepts 
were approved by a special education subject matter expert and then prototypes were developed and tested. 
In the next step, feedback was collected to improve prototypes. This process continued until a satisfactory 
body movement game is developed. In addition, we also tested and improved usability testing approach for 
body movement games for special education children. The designed game has two sections: 

2.1 Training Section 

Which was designed to teach how children can interact with Kinect camera and the game. Training section is 
composed of 6 levels which were designed with considering the level of the children’s disabilities. First 
level’s aim is to familiarize the children with the Kinect environment. Second and third levels teach the depth 
concept in a virtual environment. In 3 rd level there is not any verbal cues. The fourth level aim is to teach the 
correct hand movements during the cleaning process. The fifth level provides an environment which children 
clean the dirty carpet by body movements. The sixth level aims to teach to distinguish garbage from 
non-garbage. This level is designed to teach the children how to clean the carpet and grab the garbage and put 
them into the garbage can. Verbal cues help children to learn the skills. Figures 1-5 show a preview of the 
training section. 
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Figure 1. First Level of Training Section Figure 2. Second & Third Levels of Training Section 



Figure 3. The Fourth Level of Training Section Figure 4. The Fifth Level of Training Section 





Figure 5. The Sixth Level of Training Section 


Figure 6. A Preview of The Evaluation Section 


2.2 Evaluation Section 

In this step, user’s performances can be assessed. A user can use whatever he/she has learned from the 
training section to finish the required tasks. The evaluation section consists of one level. This level is the final 
part of the vacuum cleaner game. The evaluation is done without any intervention to check the child’s 
performance results. Figure 6 shows the evaluation level, in this step, it is expected from the children to grab 
all non-garbage objects and put them in proper places. The evaluation was accomplished without any verbal 
cues. By activating the audio button in this level, verbal cues are active then the level can be used as a 
training level too. 

Verbal cues are used more in the basic levels of training part and they are reduced through the following 
levels. Reinforcement and feedback are also used in the training part to increase the motivation and usability. 
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3. CASE STUDIES 

The most parts of the game were designed and developed in a private special education school in companion 
with special education students. To evaluate the final product, a case study was conducted in the same school 
and after a while to have more reliable results another case study was conducted in a public special education 
school. After one week, the participant’s performances in public special education schools were assessed 
again to check the consistency of their obtained skills. 

In the case studies, usability tests were conducted along with systematic observation in the presence of a 
special education expert to determine how well children can play the game and learn the skills from the 
game. Before starting the experiment, each participant was asked to clean a real carpet by a real vacuum 
cleaner. The goal was to observe the child’s performance during the cleaning process. Children should be 
able to grab the non-garbage objects by their hands and put them in the desired place and clean the carpet by 
a vacuum cleaner. The aim of this process is to check whether children can detect and distinguish the garbage 
from the other objects or not and whether they can clean the whole carpet precisely. If a child knows the skill, 
the experiment should be stopped, otherwise, the experiment continues. Those who could not do the task 
completely were accepted as a participant of the study. In the first special education school, there were fifteen 
children participated in the usability test process. Eight of them met our conditions. 

Children were asked to vacuum a furnished room by a real vacuum cleaner (Figure 7(a)) in presence of 
an expert teacher and then play the digital vacuum cleaner game (Figure 7(b,c)) in the suitable environment 
which does not distract their attention. 

Researcher watched and took notes and recorded the events while the children were performing a list of 
tasks by playing the game and being tested. 



Figure 7. (a) Child is Going to Clean the Carpet. (b,c) Children Playing the Game 

Post-test questionnaires were also used to gather feedback from the expert teacher on the game. Figure 8, 
shows photos of the children while playing the game. 

The video was recorded from each test to be analyzed later. This process iterated for 12 weeks and the 
results of each iteration were gathered. The study continued by 5 participants, one of which was an autistic 
child and one with Down syndrome. The other three were with mild MR. 

In the second case study, which was conducted in a public special education school, there were 34 
students. They suffer from either mild MR or Down syndrome. All the children who participated were 
between 8 and 24 years old. Six children had been familiar with the vacuuming skills before. Nine children 
were qualified and their performances were satisfactory to be evaluated in this study. Among the qualified 
children, there were 7 boys and 2 girls. The first evaluation was conducted and the second evaluation started 
8 days after the first evaluation. The aim was to test the children’s skills over the time. On the evaluation day, 
just 6 of the 9 participants were available at the school. All 6 participants were told to vacuum the messy 
carpet by vacuum cleaner machine. The perfect result was to vacuum the whole carpet by moving their hands 
forward and backward, and grabbing the objects and put them in their own desired places. 
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4. RESULTS AND DISCUSSION 

The process of teaching to special students, specifically mentally disabled people is very complex and takes a 
considerable amount of time. Every skill should be divided into the simplest steps before teaching to the 
mentally disabled children. The compatible vacuuming skills with Microsoft Kinect device which are 
essential to teaching to the mentally disabled children are divided into 4 steps: 

1. Children take the vacuum cleaner handle correctly. 

2. Vacuuming the carpet precisely by moving their hands forward and backward. 

3. Distinguish garbage from the non-garbage objects, and take the non-garbage objects and put 
them in their own designated places. 

4. Vacuuming the carpet precisely by walking to the right and left and forward and backward. 

These steps were also used in the evaluation section. The results were achieved based on these steps. 

It is assumed that children know all the prerequisites before starting the cleaning process. In the 
experiment, children were observed, and the experiment was continued by those children who know the 
prerequisites but do not know the task completely. Target groups for this study are the participants who: 

• Cannot do the task individually and do not know about vacuuming skills. 

• Children which know some steps but not all steps. 

• Children who know all steps of vacuuming skills separately but still do not know how to 
combine them to finish the task. 

These groups have a problem with vacuuming skill and need help. This game gave us an opportunity to 
evaluate each level separately. In each level, there is a button which turns the game sounds on or off. By 
turning the level sounds off, the level can be used for evaluation of the children. From pre-test evaluation, the 
participants who met the experiment, criteria were selected. The selected group also played all levels of the 
game individually. Corrections were considered, and the game improved and expanded based on their acts. 

4.1 The First Case Study’s Evaluation Result 

The Figure 8 shows the results of each participant’s performance in the first special education school based 
on their capability of doing the tasks separately. 
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Figure 8. The Results of the Experiment in the First School 

The plot’s Y axis shows the number of the completed tasks (Vacuuming skills which are compatible with 
Kinect technology) by the children, and the plot’s X axis shows each subject identification. It is obvious from 
the figure that, significant improvement happened on the children performances between pre-and post-test. 
Child 1A, 2A, and 4A had previous knowledge about the first two skills. After playing the game, they also 
learned all the steps. Child 3A made the major progress because he only performed the first skill in the 
pre-test, but after playing the game he learned all the steps. The results show that child 5A also had a small 
improvement. 
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4.2 The Second Case Study’s Evaluation Result 

As it is shown in Figure 9, there are nine children in the second experiment. The significant improvement 
happened to child IB, 2B, 3B and child 9B, but among them, child 2B and child 3B performances are 
admirable because they were less familiar with the vacuuming skills at the starting point. For two child 4B 
and 8B, they had a small improvement but it was not satisfactory to be considered in the results. Finally, 
child 5B, 6B, and 7B were familiar with the vacuuming process, but they had problem to grab the objects 
from the carpet. The results confirm their improvement at the post-test experiment. Seven children had an 
improvement in their experiments. 


Y ■ Pre-Test ■ Post-Test 
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Figure 9. Results of the First Evaluation 

The second evaluation started 8 days after the first evaluation. The aim was to test the children’s skills 
over the time. On the evaluation day, just 6 of the 9 participants were available in the school. All 6 
participants were told to vacuum the messy carpet by vacuum cleaner machine. The perfect result was to 
vacuum the whole carpet by moving their hands forward and backward, and grabbing the objects and put 
them in their own desired places. The children 2B, 3B, 4B, 5B, 6B, and 7B participated in the second 
evaluation. Only three children (child 5B, 6B, and 7B) did not forget the task and completed the wanted tasks 
precisely (Figure 10). To compare the first and second evaluation’s results, the number of completed tasks in 
the first evaluation and second evaluation is also depicted in Figure 10. First columns inform the first 
evaluation pre-test outcomes, second columns show the first evaluation results, and third columns depict the 
second evaluation results. 
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Figure 10. Results of the First Group’s Second Evaluation 
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5. CONCLUSION 

In this study, we designed a body movement vacuum cleaner game and tested it in two special education 
schools (Bilge Ozel Egitim Ve Rehabilitasyon Merkezi and Sait Ulusoy Oazel Egitim Uygulama Merkezi 
special education schools both located in ANKARA -TURKEY) by the help of a special education expert, 
and we found that the game is suitable for the children with mild spectrum of mental disabilities. This finding 
supports the previous studies results about the effectiveness of body movement game as a rehabilitation tools 
(Lange et al., 2012; Vernadakis, Derri, Tsitskari, & Antoniou, 2014). Moreover, it confirms that interactive 
video games are compatible with the repetitive and stereotyped behaviors of autistic children which made it 
attractive to them (Boutsika, 2014). 
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Both realistic situation test, and virtual situation test results were compared, and the conclusions are as 
follows: The observation of the all children performance reveals that the children who can interact with game 
avatar, have more chances to finish all experiments, which shows that utilizing video games in special 
education have shown positive effects in the children learning (Gonzalez, Cabrera, & Gutierrez, 2007; 
Ruggiero, 2013). These findings support the current study findings, as the results of the study indicate, those 
children who could interact with the main avatar, finished the task completely and precisely. Those children 
who could not finish the wanted tasks mostly had problem in making interaction with the avatar. 

The key factor of the proposed game is teaching the differences between garbage and non-garbage 
objects. Using appropriate body gestures increases children perception of being in the virtual world, and 
increases the personal self-sufficiency, and causes to finish the tasks by immersion into virtual reality. 

Vacuuming the room completely and precisely is the main goal of the study but some pre-request steps 
are not compatible with Kinect. Some of them are not feasible to design because of the Kinect calibration 
problem in some conditions, and others are the minor steps which can be learned by the simplest technics. 
General body movements in the 3D environment were the strength reason for using Kinect. 

The findings have shown that gesture based games may have a great potential in the education of special 
education children. Applying the Microsoft Kinect in education is impressive and increases the children’s 
creativity and interaction in the class (Hsu, 2011). Some researchers inform the potential of using a 
gesture-based Kinect game to communicate with deaf people and confirm the positive effect of playing in a 
group for deaf people (Soltani, Eskandari, & Golestan, 2012; Zafrulla, Brashear, Starner, Hamilton, & Presti, 
2011). Their finding supports the current study findings which children motivation is increased while playing 
vacuum cleaner interactive game. 

The current study is a unique one because: 

(a) The target groups of the study are the mentally disabled children with different disabilities. 

(b) It is a comprehensive study of using Microsoft Kinect in special education. 

(c) It focuses on teaching vacuuming skills to children with mental disabilities. 

Some of the children had physical disabilities too. The proposed vacuum cleaner game needs to be played 
by using body and hands movements which may have some positive results in their physical movement 
therapies. Hung, Chang & Han (Hung, Chang, & Han, 2016) investigated the use of Microsoft Kinect in 
increasing the motor control in children with cerebral palsy, and their research results indicate that the 
significant improvement was observed in three participants. 

Finally, considering the results, the proposed vacuum cleaner game had a positive contributes to children 
learning by fun, especially for those with a mild spectrum of the mental disabilities, and it is a big step 
forward for teaching basic daily life skills to the mentally disabled children in an efficient manner. 

5.1 Limitations of the Study 

With respect to the high-tech devices which were used in this study, there are many areas which technologies 
do not support yet such as Kinect camera does not detect the body joints if there was any overlap. It causes 
wrong calibration. This supports Hsu (2011) findings of the needing re-calibration when students walk out of 
the range that Kinect can reach or when calibration is not done correctly. Some parents are not familiar with 
the computer, considering their level of knowledge is also important to design a user-friendly and easy to use 
the game. Finding the proper participants and taking permission from their parents to do the usability test, 
were also a big challenge. 
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ABSTRACT 

Massive open online courses (MOOCs) describe platforms where users with completely different backgrounds subscribe 
to various courses on offer. MOOC forums and discussion boards offer learners a medium to communicate with each 
other and maximize their learning outcomes. However, oftentimes learners are hesitant to approach each other for 
different reasons (being shy, don’t know the right match, etc.), hi collaborative learning contexts, the problem of 
automatic formation of effective groups becomes increasingly difficult due to very large base of users with different 
backgrounds. To address this concern, we propose an approach for group formation of users registered on MOOCs using 
a modified Particle Swarm Optimization (PSO) technique which automatically generates dynamic learning groups. The 
algorithm uses the profile attributes of users in terms of their age, gender, location, qualification, interests and grade as 
the grouping criteria. To form effective groups, we consider two important aspects: a) intra-group heterogeneity and b) 
inter-group homogeneity. While the former advocates the idea of diversity inside a particular group of users, the latter 
emphasizes that each group should be similar to one another. We test our algorithm on synthesized data sampled using 
the publicly available MITx-Harvardx dataset. Evaluation of the system is based on the fitness measures of groups 
generated using our algorithm which is compared against groups obtained using some of the standard clustering 
techniques like k-means. We see that our system performs better in terms of forming effective learning groups in the 
context ofMOOCs. 


KEYWORDS 

Group Formation, MOOCs. Online Learning 


1. INTRODUCTION 

In many collaborative learning contexts, students are organized into small groups to complete their tasks with 
a common group related goal (Lou, 2008). During the past decades, hundreds of studies have been made to 
investigate the effectiveness of collaborative learning. Most of these studies conclude that well-constructed 
learning groups can effectively drive teamwork among the members of the group and can have better 
performance than poor-constructed groups (Shimazoe, 2010), (Deibel, 2005). Moreover, there are studies that 
tell us that the conventional approaches of grouping students together based on self-selection or 
random-selection are not well suited in educational domain (Shimazoe, 2010). Group formation in education 
essentially requires a broader study of students’ backgrounds, their traits and a know-how of the instructional 
environment. 

The emergence of Massive Open Online Courses (MOOCs) as a major source of learning in the modern 
world has created several challenges in terms of forming effective groups of learners. More people signed up 
for MOOCs in the year 2015 than they did in the first three years of the ‘modern’ MOOC movement (which 
started in late 2011 - when the first Stanford MOOCs took off) (Shah, 2015). The students registered on 
MOOCs have varied demographics in terms of the countries they originate from, languages they speak and 
their personality traits. Moreover, studies show that the lack of effective student engagement is one of main 
reasons for a very high MOOC dropout rate (Onah, 2014). Although many thousands of participants enroll in 
various MOOC courses, the completion rate for most courses is below 13%. Further studies (Lou, 2008), 
(Shimazoe, 2010), (Zepke, 2010) have been made to show how collaboration or active learning promotes 
student engagement. Hence, we believe that forming effective learning groups of students would foster better 
collaboration and could also help mitigate the dropout rates to some extent. 
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Keeping the above in mind, our work focuses on exploring the possibilities of assisting MOOC 
learners in the process of self-organization (e.g. forming study groups, finding partners, encourage peer 
learning etc.) by developing a group formation strategy based on predefined set of user attributes like 
age, gender, location, qualification, interests, grade etc. We use a modified particle swarm optimization 
(Kennedy, 2011) technique that helps in effective group formation by looking at the different user 
attributes along with the grouping conditions of intra-heterogeneity and inter-homogeneity. The idea is 
to form learning groups that are diverse internally while being similar to each other on certain aspects, to 
have the best possible learning outcomes. 

The remainder of the paper is organized as follows. Section 2 outlines the proposed model and data 
for generating effective groups using the modified particle swarm optimization technique. In section 3, 
experimental evaluation and results are presented. Finally, Section 4 ends with a conclusion and future 
work. 

2. PROPOSED METHOD 

We look at the data model along with the design and description of the group formation algorithm. 

2.1 Data 

The data used in our research comes from the de-identified release from the first year (Academic Year 
2013: Fall 2012, spring 2013, and summer 2013) of MITx and HarvardX courses on the edX platform 
(HarvardX-MITx, 2014). These data are aggregate records, and each record represents an individuals' 
activity in one edX course and contains many diverse information about the profile of the learner 
(e.g. age, gender, location, qualification, grade etc.). For our analysis and without loss of generality, we 
selected records with attributes about age, gender, location, qualification and grade. Moreover, we 
enhance this information with synthesized data about learners' interests. This information is not 
available via the mentioned dataset but is potentially useful for creating effective groups. A brief 
overview of the dataset attributes can be found in Figure 1 along with a sample of our dataset in 
Figure 2. 


Attribute 

Short 

Type 

Comment 

user id 

id 

Numeric 

Unique identifier 

age 

age 

Numeric 

Calculated using year of birih M(ale)/ 

gender 

gen 

Binary 

F(emale) 

location 

loc 

Categorical 

City of the learner 

qualification 

qua 

Ordinal 

5 levels 

interests 

int 

Hierarchical. Categorical. Multi-Value 

Info about learners’ interests graded 

grade 

grade 

Numeric 

between 0(Min) and l(Max) 


Figure 1. Data Attribute Description 


id 
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gen 

loc 
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int 

grade 
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32 
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Doctorate 

ML 

0.1 
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28 

M 

Berlin 

Secondary 

AI 

0.4 
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27 

F 

Edmonton 

Bachelors 

Science 

0.8 

4 

22 

F 

Las Vegas 

Masters 

Soccer. AI 

1.0 


Figure 2. Dataset Sample 
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2.2 Data Modelling 

A description of how each attribute in Figure 1 is modeled, is as follows: 1) age: age range of the users' 
are segregated in these five bands: less than 20, 20-25, 25-30, 30-35, 35 and above. 2) location: location 
attribute is categorized into three options: same city, same country or same time zone. 3) gender: male 
or female gender options. 4) qualification: the qualification attribute has been divided into 5 levels: less 
than secondary, secondary, bachelors, masters and doctorate. 5) interests: the interest attribute contains 
one or more values about learners' interest. 6) grade: the grade attribute has averaged learners' grade 
from previous courses, between 0(min) and l(max). 

A sample of data vectors can be seen in Figure 3. The 'x's in the table represent null value. It must be 
noted that not all six attributes are required to be used for any kind of grouping. Our proposed 
algorithm is flexible enough to take one or more of these attributes for group formation. Moreover, we 
can tune the way each of these attributes contribute in group formation in terms of intra-group 
heterogeneity and inter-group homogeneity. For instance, a reasonably heterogeneous group would refer 
to a group where student-grades reveal a combination of low, average and high student-grades. This is 
justified by the recommendation of Slavin (Slavin, 1987) who proposed that students should work in 
small, mixed-ability groups. Hence, it is necessary that grade distribution is even across all groups i.e. 
the average grades of students across all groups should be same (inter-group homogeneity) while 
maintaining that within each groups the grades are diverse (intra-group heterogeneity). 


userid 

age 

gen 

loc 

qua 

int 

grade 

1 

30-35 

M 

same city 

>= Masters 

X 

0.1 

2 

X 

X 

X 

Bachelors 

Football 

0.4 

3 

25-30 

F 

X 

X 

X 

0.8 

4 

<=25 

X 

same timezone 

<=Bachelors 

X 

0.1 


Figure 3. Sample Data Vectors 

Another important factor for group formation in collaborative learning is the interest of group 
members since it has the potential to change the involvement of individuals in learning (Freeman, 
2014). A group with common interests will have more interactivity and discussions that is likely to 
make the learning process more engaging. The same can be said about the 'location' attribute. Students 
residing in the same city, country or time zone will be able to collaborate better due to minimal time 
differences. 

2.3 Algorithm 

In this section, we discuss our group formation algorithm in detail. In short, at first we use a modified 
K- means clustering algorithm (Hartigan, 1979) to fit our data attributes to seed initial swarm of particles. 
Then we use a hybrid particle swarm optimization technique to build the final group of learners. 

2.3.1 Modified K-means 

In modified K-means algorithm, at first all the cluster 'centroids' or 'mid-points' are randomly initialized 
using the data vectors. Then the distance for each data vector is calculated using a scoring system 
wherein the distance between each attribute of a data vector to that of its corresponding attribute of all 
centroids is calculated. The data vector is then assigned to that cluster where it has the least distance'd' 
with its corresponding centroid as per the equation in the figure below: 

N d 

d(z p , rrij) = \/{^2{z p k - rrijk) 2 ) 
k =l 


Figure 4. Distance Calculation 
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where k represents a particular dimension or an attribute, Nd denotes the input data dimension 
(number of attributes), Nc denotes the number of centroids of the clusters or the number of clusters to be 
generated, zp denotes the p-th data vector, mj denotes the centroid of cluster j. 

The attributes are modelled in the following way for distance calculation: 1) age, qualification: age 
and qualification attributes are divided into levels in such a way that adjacent levels have a distance of 
one unit. The distance is then normalized in range [0 - 1] by dividing it by the maximum distance value 
possible. 2) gender, location: For any given categorical options of gender and location, if the values for 
any two users are same then the distance is 0 else 1. 3) interests: The hierarchy we used for interests of 
users is based on WordNet (Miller, 1995) and the similarity measure used is based on the Wu and 
Palmer method (Wu, 1994) score that considers the depths of the two synsets in the WordNet 
taxonomies, along with the depth of the LCS (Least Common Subsumer). Score for this similarity is 
between 0 and 1, since we are implementing our system in a distance measure (and not similarity) the 
final value of distance between the interests is [1 - score]. 4) grade: For grade attribute, distance 
measure between a data vector and centroid is simply the difference between their grade values. 

In traditional k-means (Hartigan, 1979) algorithm, the centroids are typically recalculated by taking 
the average sum of all the data vectors present within a cluster until a stopping criteria is reached. 
Flowever, in this case, centroids are recalculated in a different way based on each attribute value of 
every data vector within a particular cluster, as per the following rule: 

1) age, grade: centroid value corresponding to these attributes is the mean of age and grade of 
every data vector present within the cluster. 

2) location, gender, qualification, interests', centroid values for each of these attributes is the most 
common attribute value within the data vectors belonging to a particular cluster. Moreover, 
K-means clustering process ends when any one of the following stopping criteria is reached: when 
the maximum number of iterations has been exceeded or when there is little to no change in the 
centroid vectors over multiple iterations. We use k- means for two different purpose: 1) To 
formulate baseline clusters to compare against the clusters or groups generated using hybrid PSO 
algorithm and 2) To initialize one of the particles used in the hybrid PSO algorithm. We use two 
different baseline models for result comparison, as mentioned below: 

1) Number of clusters/groups (k) is specified: In this case, the number of clusters to be formed 
using k-means is specified by the user. Each cluster obtained after running k-means will have data 
vectors that are very similar to each other. However, in order to have intra-cluster heterogeneity 
we need to have diverse data vectors within a cluster. To build an unbiased baseline model, we 
create equal number (k) of empty clusters. Then using the first cluster obtained via k-means, we 
evenly distribute the data vectors in them to each of these empty clusters. We repeat this process 
with all other data vectors from the clusters obtained using k- means. In the end. we have a new 
set of clusters with data vectors, which are diverse and can be used as a good baseline for result 
comparison. 

2) Number of users (a) in a cluster/group is specified: In this case, the number of users in each 
cluster or group is pre-decided. In order to account for intra-cluster heterogeneity, we create 
empty clusters, each with size a. Every cluster is then filled with data vectors obtained from each 
of the clusters generated using k-means until a value is reached. In the end, we have new set of 
clusters (size a) with data vectors that are diverse and can be used as a good baseline for result 
comparison. 

Next, we discuss the hybrid particle swarm optimization (PSO) algorithm. We modify the standard 
PSO algorithm for MOOCs and combine it with modified k-means to build a hybrid algorithm for group 
formation. 
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2.3.2 Hybrid Particle Swarm Optimization 

Over several years, the particle swarm optimization (Kennedy, 2011) has been used to solve various 
problems of the level of complexity NP-Hard (Jarboui, 2008), (Yin, 2006). The results of these studies 
show that PSO has been very effective in solving problems of this level of complexity. Our problem 
involves optimization of different student attributes, hence we used hybrid PSO to form effective 
learning groups. The aim of hybrid PSO is to find an optimum solution based on a certain fitness 
function. Every particle is evaluated with respect to this fitness function; the fittest particle is accepted 
as solution. In hybrid PSO, we calculate the velocity and position of all particles after every iteration 
based on the equations below: 

Vi,k(t + 1) = wvi'k(t) + ciri t k(t)(pBest itk (t) - x^ k (t)) + C 2 r 2 , k (t)(gBest itk (t) - x i>k (t)) 


Xi(t + 1) = Xi(t) + Vi(t + 1) 


Figure 5. Velocity and Position Equation 

where xi is the current position of the particle, vi is the current velocity of the particle, w is the inertia 
weight, cl and c2 are the acceleration constants, rl,k(t), r2,k(t) are random numbers between (0,1), and 

k= 1,., Nd- 

In the context of grouping, a single particle in PSO represents the Nc cluster centroid vectors, wherein 
each particle xi is constructed as follows: 


xi = (mi 1,.my., m?7Vc) 

where mij refers to the j-tli cluster centroid vector of the i-th particle in the cluster Cij . 

Therefore, a swarm represents a number of candidate solutions, as each particle in itself is a solution. We 
use the modified k-means to initialize the Nc centroid vectors of one of the particles of the swarm. The 
centroid vectors of remaining particles are initialized randomly using the data vectors. Once the groups of all 
particles are initialized, we calculate the fitness of each particle, which is measured using the following 
fitness error functions: 


fitness(P igrade ) = VcyeJV,. : \max { Ci jgrade ) - min { c ijgrade ) 

T l f= 1 \E^C ij ,d(z p .m j )/\C ij \] 


fitness(P iloc , nt ) = 


N r 


Figure 6. Fitness Equations 

where ‘d’ is Euclidean distance defined in figure 1, ICijl is the number of data vectors belonging to group 
Cij. 

Above-mentioned equations are fitness measures of a particle in terms of 'grade' and ['location', 'interest'] 
attributes respectively. The less the fitness error, the better the quality of groups formed. More specifically, if 
the grade difference between the max and min grade value for all groups within a particle is less than a 
threshold t (t=0.1), the particle is fit. We select the gBest (global best) and the pBest (personal best) particles 
based on the combination of fitness achieved using equations in Figure 6. The particle with least 'grade' 
difference and minimum 'location' and 'interest' distances, is selected as the global best. In addition, each 
particle stores its local best state, which has the least grade difference and minimum 'location' and 'interest' 
distances, in any given iteration. 

Next, we update the group centroids of each particle using equations in Figure 5. However, the update for 
each attribute of the centroids is different from each other. In case of age and grade attributes, the updated 
values depend on the age, grade values of global best and personal best particle whereas for all other 
attributes [location, gender, qualification and interests], the updated values depend on the most common 
values of all data points in respective groups. This entire sequence completes one iteration of the algorithm. 
PSO is usually executed until a specified number of iterations has been exceeded or if a certain level of 
fitness has been achieved. 
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Below is the summary of group formation using hybrid particle swarm optimization (PSO): 

1. Initialize each particle with Nc randomly selected cluster centroids, except one centroid which is 
initialized using modified k-means. 

2. for iteration t = 1 to tmax 

(a) for each particle i do 

(b) for each data vector zp 

i. Calculate the attribute distances d(zp,mij ), between the data vector zp with each 
cluster centroid mij, for all cluster centroids Cij. 

ii. Assign zp to the Cluster Cij where the distance is minimum. 

iii. Calculate the fitness of particle using equations in Figure 6. 

(c) Update the global best particle in the swarm along with the personal best of each particle. 

(d) Update the group centroids of each particle using equations in Figure 5. 

3. EXPERIMENTS AND RESULTS 

Our experiments employed a series of testing to analyze the effectiveness of the PSO algorithm for 
group formation in MOOCs. We compare the quality of clusters generated using the modified k-means 
and hybrid PSO based on the calculated fitness error as defined in equations in Figure 6. The objective 
is to help us measure diversity inside each of the group while at the same time making sure that every 
group is similar to the other based on the grading levels. 

The hybrid PSO algorithm was run on a computer with 2.7 GHz Intel Core i5 processor and 8 GB 
RAM. In order to examine the effectiveness of the PSO, five different sets of data for each [100, 1000, 
5000] samples were generated randomly from the original dataset that has around 300k records. The 
parameters used for velocity update (refer equation 2) are, w = 0.72 and cl = c2 = 1.49. These values 
were chosen to ensure good convergence (Bratton, 2007). In addition, the number of particles predefined 
is [10, 20, 50] respectively for data with volumes of [100, 1000, 5000] records. This was chosen based 
on the study (Chen, 2010) that any number of particles between 10 to 100 are capable of producing 
results that are clearly superior or inferior to any other value for a majority of the tested problems. The 
results reported is averaged over five different simulations, each simulation was run with different data 
samples. Our results will be analyzed on two different baseline models: 1) Number of groups (k) is 
specified and, 2) Number of users (a) in a group is specified. 

3.1 Results 

Figure 7 below shows the effect of varying the number of groups on the fitness values for 'grade 
difference', 'interest' and 'location' distances for 100 data records. As expected, the fitness error should 
go down as the number of groups increase. We calculate the grade fitness based on equation in Figure 6, 
wherein the difference between the maximum and minimum grade values is taken from all the groups. 
This difference is represented as the fitness score in figure 7 (a). It is seen that the fitness score 
decreases with increase in number of groups that means that the quality of groups formed increase as the 
number of groups increase. Next, we calculate the 'location' and 'interests' fitness based on equation in 
figure 6. The total distance for 'location' and 'interest' is normalized to produce a fitness score that is 
shown in Figure 7 (b). A similar pattern is seen wherein the fitness score decreases with increase in 
number of groups. 
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Figure 7. Effect of Different Number of Groups on Fitness 

We also compare the fitness results when the number of users in a group (a) is predetermined; the 
results are shown in Figure 8. Looking at the grade fitness graph (Figure 8 (a)), the fitness error decreases 
with increase in the number of users per group. This is expected because with more users the chances of 
'grade' scores being skewed decreases, hence the grade fitness increases. 

However, the grade, location and interests’ fitness for the hybrid and baseline model is close for low 
values of (a). This can be attributed to the fact that with lower number of learners in a group, the 
chances of similar values for the mentioned attributes within a group decreases. 

Similarly, for 'location' and 'interest' fitness (Figure 8(b)), the hybrid PSO models performs the same 
as the baseline model when the number of users per group are less. However, it outperforms the baseline 
when the number of users per group increase. However, the overall fitness error may increase even with 
the increase in number of users. It can be seen that the fitness score increases from 0.43 to 0.48 when the 
number of users per group increase from 20 to 25. 





(a) Grade Fitness (b) Fitness for Location, Interests 

Figure 8. Effect of Number User Per Group (A) on Fitness 

Overall, the results show that the hybrid PSO model outperforms both the baseline models for 
generating better quality groups. Although, the algorithm could not be tested real-time on an actual 
MOOC platform, these results nevertheless provide promising insights when applying hybrid PSO 
technique in group formation using student attributes. Hence, it would be worthwhile to integrate it 
within an actual MOOC to get a realistic opinion on its performance. 
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4. CONCLUSION AND FUTURE WORK 

In this paper, we presented a framework using a hybrid particle swarm optimization to form student 
groups based on attributes like [age, gender, location, qualification, interests and grade]. The evaluation 
of the proposed algorithm was done in the previous section to determine the overall quality of groups 
formed in terms of fitness. The results showed that the group quality was better when compared to the 
baseline model of groups formed using the modified k-means method. The proposed strategy can help 
the instructors to automatically generate suitable learning groups of students for online classes, which 
may foster better collaboration between the participating students by increasing their level of interaction 
with like-minded and diverse population. 

As future work, we plan to conduct tests on an actual MOOC platform to get a real-time assessment 
of the quality of student groups formed based on the proposed algorithm. The algorithm can also be 
improved to add more attributes that could potentially increase the chances of forming better quality 
groups. These attributes could be derived based on the past courses that the students had registered for, 
or i n some form of a feedback from students themselves based on a certain questionnaire. Case studies 
reveal that the number of participating users in MOOCs is increasing every year, hence it becomes quite 
challenging to establish the same kind of communication that exists within a classroom. However, by 
using hybrid PSO to generate dynamic learning groups, we believe we can bridge that gap to some 
extent. 
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ABSTRACT 

There are many difficulties with learning and teaching programming that can be alleviated with the use of software tools. 
Most of these tools have focused on the teaching of introductory programming concepts where commonly code fragments 
or small user programs are run in a sandbox or virtual machine, often in the cloud. These do not permit user software to 
directly interact with system resources and accordingly do not directly support advanced programming concepts such as 
multiprocessing, inter-process communication (IPC), Device 10, concurrency, synchronization, and platform 
independence. This paper presents a new tool for teaching advanced programming called the Interactive C Environment 
(ICE) that is designed to support teaching of all these programming concepts by automating many tasks for both the 
learner and teacher. It consists of an integrated development environment (IDE) with a built-in editor, compiler, 
automated testing framework, and a built-in makefile processor. It supports programs written in C-99 and a subset of C++ 
using both MS Windows and Linux System APIs. ICE not only provides secure, automated testing and assessment of 
projects but also group learning support in terms of comparative statistics. 


KEYWORDS 

C-programming, Programming Based Learning, Automated Assessment 


1. INTRODUCTION 

The importance of teaching programming within the computer science discipline and the associated learning 
difficulties is well recognized (Thomas et.al, 2010; Robins et.al, 2003). A constructivist approach (Jonassen 
2003) to teaching programming is commonly taken in the form of what some called “active” or experiential 
learning (Kolb 1984), where students learn by practising problem solving skills through writing software. 
Many tools have been developed to assist in both teaching and/or assessing programming skills with some 
success. Some tools such as Codingbat (Stanford 2011) focus on providing a simplified environment (IDE) 
for representing problems to students with immediate feedback. Other tools (Ahomiemi and Reinikainen, 
2006; Ala-Mutka 2005, Whalley 2011) focus on automatic assessment of student work and a third class of 
tools, such as MOSS, JPLAG and various others, are specialized for plagiarism detection (Aiken 1994; 
Prechelt et.al, 2002; Pawelczak 2013; Lancaster and Culwin 2004). Further, some tools provide a 
combination of these capabilities. Many of the tools in the first and second category are focused on teaching 
introductory programming. Some tools such as Alice (Cooper 2001) and Scratch (Marji 2014) focus on 
teaching programming theory without the semantics of production languages through the use of graphical or 
visual programming metaphors (Kelleher 2005; Johnston 2004). Others focus on teaching the syntax and 
semantics of production languages as well as simple algorithms and the application of this knowledge 
towards solving simple computational problems (Pullan 2013). Commonly, in these cases the user programs 
are run in a sandbox environment or virtual machine (Pawelczak and Baumann 2104) where software 
exceptions are caught by the tools and handled in a safe manner thereby avoiding students having to deal 
with the unexpected consequences of poorly written code which might confuse a beginner. Since sandboxing 
isolates the user program from directly accessing the hardware and operating system resources, this approach 
inhibits the use of advanced programming concepts such as multiprocessing, concurrency, synchronization, 
inter-process communication, device IO and platform specific dependencies. Accordingly these tools are, in 
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general, not suited to teaching and learning in advanced programming courses such as systems programming 
and network programming or computationally efficient, low level programming. 

To address the issue of supporting teaching and learning of more advanced programming the Interactive 
C Environment (ICE) is specifically designed for the teaching of systems and distributed programming. ICE 
provides automated mechanisms for self-assessment and feedback for programs that goes beyond just 
teaching language semantics using relatively simple “sandboxed” sequential programs that do not interact 
with the underlying computing platform. ICE is being used in the Systems Programming undergraduate 
course at Griffith University. The capabilities of ICE and its support for teaching and learning are presented 
in the following sections. 


2. INTERACTIVE C ENVIRONMENT 

ICE is a client-server system. The ICE server consists of a number of remotely hosted PHP scripts that are 
responsible for overall system and user administration including validating registered users, managing 
available exercises and test harnesses, providing the appropriate set of exercises for individual users, logging 
user activity including assessment and performance scores communicated to it by the client, performing 
plagiarism checking and returning statistical information of the performance of the user relative to other 
users. The ICE server communicates with ICE clients by tunnelling over the HTTP protocol using a cipher 
block chaining encryption scheme. 

The ICE client consists of an integrated IDE with a built-in editor, compiler, an automated testing 
framework, and a built in makefile processor. Versions of the client exist for both MS Windows and Linux. It 
supports programs written in C-99 with inline assembly and a subset of C++. Irrespective of the host 
operating system the client supports user programs that target both MS Windows and Linux System APIs. 
The client also provides the user with performance feedback in the form of a visual representation of each 
user’s standing in terms of comparative statistics with respect to the other members of particular courses. 

In contrast to many online tools, the ICE client software executes compiled user programs natively on the 
local computer hardware granting them direct access to all the hardware and operating system resources of 
the underlying platform. ICE provides special support to test programs that make use of these enhanced 
capabilities. The ICE client encrypts and stores all the user code and test harnesses locally. Accordingly, it 
can run in either online or offline mode, with the limitation that while offline it cannot update the available 
exercises or user performance statistics. The ICE user interface consists of a window divided into three main 
panels: the exercise browser panel on the left, the editor panel on the right and the console panel beneath it. 

2.1 ICE User Interface 

In contrast to the many Integrated Development Environments (IDEs) that support hundreds of features and 
settings, which can be confusing to the novice user, the ICE user interface is designed to be as simple as 
possible. The program window (Figure 1) is divided into three main panels: the exercise browser panel on the 
left from which users select what problem to work on. the editor panel, which is the primary work area, and 
the console panel beneath it, which is used for interacting with users’ programs. At the bottom left of the 
window is a prominent button labelled “Build” which activates the entire tool chain when clicked: it saves 
and compiles the source code, runs the applicable tests, reports the results to the ICE server and obtains 
updated statistical information about the user’s performance. 

The exercise browser panel consists of a drop down list of courses at the top. Below that is a collapsible 
tree widget, listing available exercise questions, followed by a prominent build/execute button and beneath 
that, a fde browser subwindow showing a list of files in a project appears when required for multi-file 
projects. The tree widget also shows a visual depiction of the completion status of each exercise. An empty 
box indicates an exercise has not been attempted. A ticked white box indicates the exercise was satisfactorily 
completed. A red crossed box indicates the program failed static and syntactical tests. A white crossed box 
indicates the user program passed all of the static and syntax tests and compiled successfully but completely 
failed the functional tests. A white box with an ‘o’ indicates that the user’s program passed the majority of 
functional tests but did not fully satisfy the task’s requirements. 
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When the user first logs into ICE, it attempts to communicate with the ICE server. If successful, the 
course list box at the top of the exercise browser is populated with the courses that the user has access to. 
Selection of a given course from this list will populate the tree widget beneath it with the appropriate 
exercises grouped into (weekly) sets. If the ICE server cannot be contacted, the user only has access to the 
exercises, test harnesses and data that was accessed in previous online sessions. Also, ICE will store any user 
generated data that would normally be interactively sent to the server until the next time that ICE is in online 
mode when it will send the bundled stored data to the server. 




Figure 1. ICE Main Window for a Single File Project Showing Figure 2. ICE Main Window with a Multi-File Project 
User Program Showing Makefile Content and File Browser 


Selecting an exercise from the exercise browser will cause the editor panel to be loaded with relevant 
boilerplate/template code at the first instance, and thereafter by the user’s updated code. If the exercise is 
predefined to be a multi-file project, the editor panel is loaded with a relevant makefile template and the file 
list subwindow is made visible (Figure 2). Once the project dependencies have been defined in the makefile 
and the project is ‘built’, the nominated C source files are then created and displayed in the file list 
subwindow. Users can load any particular project file into the editor by selecting them in the subwindow, 
which automatically saves the currently loaded file. 



Figure 3. ICE Client Functions 

When an exercise is built any compile or make errors will be reported in the console panel. Clicking on 
any error with the mouse will load the relevant source code file into the editor if it is not already loaded, 
highlight the relevant line and place the cursor on it. After successful compilation the user’s program is 
automatically run in user mode (without test harness if it is not a program fragment) with the standard input 
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and standard output redirected to the console panel. If appropriate, the user can supply command line 
arguments for their programs using a dialog box. If a test harness is present, the program will be run again, 
possibly multiple times depending on the test conditions, under the control of the test harness. Each time a 
build is attempted telemetry data will be sent to the server consisting of the user program’s encrypted source 
and object code and the test and execution time scores. These functions and processes implemented by the 
ICE client are depicted in Figure 3. 

2.2 Single and Multi-file Programming 

ICE supports three basic types of programming exercises. Those that consist of code fragments, complete 
single file programs and multi-file programs. Exercises that require code fragments to be written typically 
involve students creating or correcting code consisting of a single function, data structure or macro. These are 
always executed within predefined test harnesses that are invisible to the user. This class of exercise is 
typically used for students to focus on, and master, specific programming constructs and algorithms without 
needing to consider supporting functions that might be required to exercise those constructs or algorithms. 

Single file exercises are small standalone programs with a defined main entry point. These are either 
executed independently, possibly taking command line arguments, or they are executed within a predefined 
test harness. This class of exercise is typically used for students to master a wide variety of more complex 
aspects of programming that are highly focused and require larger programs to be developed. 

Multi-file exercises require the users to define the project’s dependencies in a makefile but are otherwise 
handled in the same way as single file exercises. This permits students to master the concepts and use of 
traditional automated build systems and to develop much more comprehensive software programs. 
Additionally, multi-file programs permit the creation of multi-process programs by defining each required 
child process of the main process as a separate makefile target. 

2.3 Automated Standard Testing 

ICE supports a variety of automated white-box and black-box testing features. At the simplest level the client 
can impose syntax restrictions on the user’s programs. These involve Boolean algebraic expressions 
containing specific program symbols that either must be or are not permitted to be utilized in the user 
program. The program will not be compiled unless these restrictions are followed. This permits certain 
software solutions to be enforced or prohibited to ensure that students gain experience in practicing specific 
skills to create alternate solution forms and encourage divergent thinking (Plucker 2010). Syntax errors and 
common semantic errors are detected and reported during the compilation phase. 

At the next level, ICE can repeatedly apply black-box testing without the use of a test harness by 
evaluating the standard output of a program in the absence of, or in response to, a series of supplied 
command line input arguments. One problem with automated blackbox testing is that once users determine 
the specific output that will be tested for, they can create workarounds that disregard exercise requirements 
and instead simply produce the expected test output. Having a series of tests with different input arguments 
means that a program that is created to produce a fixed output will fail all but the one input condition it is 
designed to give the correct output for. At this level the testing regime is, by necessity, fairly simple: is the 
output correct or not for given input conditions? Accordingly, testing reports specific failures to the users 
such as “got 1,2,3 but expected 5,4,3” and scores the supplied solution to the exercise accordingly in 
response to whatever tests have been run on the user program. 

The most comprehensive testing capability is provided through the use of test harnesses. These broaden 
the scope of what can be tested, beyond textual output in response to static input. In addition to permitting 
command-line arguments to be adjusted dynamically, they permit the dynamic injection of data into the 
standard input stream and any other IO streams used by running program. This permits a test harness to 
simulate user interaction with a program under test or manipulate the user program’s interaction with the 
file-system. This feature allows testing to assess a user program’s handling of human computer interaction 
issues and its interaction with dynamic file systems. Test harnesses also have access to all of a program’s 
output streams such as standard output, files and display buffer and can analyze them for correct operation. 

ICE test harnesses also allow white-box testing of a user program by permitting individual, pre-specified 
user functions to be inspected from inside and out. From the outside, pre-specified functions can be invoked 
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in isolation from the program within an environment created specifically for the purpose by the test harness 
to test their handling of boundary and random conditions. Inspection from within is performed by function 
injection: the remapping of symbols used by the user program to another defined within the test harness itself 
or injection library, e.g. intercepting calls to printf() and redirecting them to myprintf(). This permits the test 
harness to inspect the memory and file structures created by a program either before of after a pre-specifted 
function call ensure that they are correct and assess any side effects the function might have. 

One of the issues with automated testing is that the test harness often includes a reference solution that the 
user code is tested against for various input conditions. Without a reference solution, test harnesses are 
limited in the scope and range of testing that can be performed on user code since they can only partially 
model the expected program behaviour. However, the reference solutions must be kept hidden from the users 
to avoid them simply copying them. The approach taken by ICE is for the server to encrypt the test harness 
so that while it is being downloaded or stored by the client it can not be deciphered by users. A related issue 
is inhibiting users from sharing their own solutions in bulk with others or having them stolen. Encrypting the 
locally stored user programs using customized per-user keys as well as all client communication also solves 
this problem. While users may copy solutions to individual exercises from the editor and paste them 
somewhere else, ICE stores them securely so other users cannot decipher them. This permits users to copy 
their solutions from one computer to another for their own use, but not for the benefit of others. 

Additionally, the ICE client evaluates the computational complexity of the user programs via measuring 
program run times. Since the programs are run in native mode on the user’s individual machine, there is 
likely to be differences based on the hardware configuration of their machines. Accordingly the measured 
results are normalized by the runtime of a benchmarking function. 

2.4 Advanced Testing Capabilities 

Some of ICE’s standard testing features can be found in other tools. Where ICE is differentiated is in its 
unique support for white-box testing of advanced programming concepts. These include handling of platform 
dependencies, multi-processing, inter-process communication and distributed systems programming. 

ICE test harnesses can select platform specific dependencies to apply irrespective of the underlying 
operating system that ICE is running on. This permits the appropriate platform specific API to be exposed to 
the user program. Currently WIN32 and UNIX variant dependencies can be individually or jointly specified. 
If jointly specified, user programs must be portable, that is, they must compile and run on both WIN32 and 
UNIX variant operating systems. ICE will then compile and run the user program twice each time applying 
the respective platform dependencies. Where system APIs are incompatible, users are required to use 
conditional compilation via preprocessor directives. A significant subset of the system specific APIs for each 
operating system is currently supported including file system, multithreading, synchronization, device IO, 
IPC, timers and process management functions. 

ICE provides support for user programs to perform process management including manipulation of 
environment variables. In regard to white-box testing of multi-process exercises, ICE test harnesses can act 
as either the parent or a child process in a solution to ensure that the corresponding process behaves correctly 
in relation to the other. In the case of process creation, the different process model used by MS Windows 
makes the Unix fork(), method very difficult to replicate. When running on MS Windows, the ICE 
implementation of fork, departs from its standard behaviour by starting the child from the main entry point, 
but then diverting execution at the fork call towards the child code. It inherits all of the resources of the 
parent process, but not the values of the automatic or global variables. 

ICE further supports testing of multiprocessing exercises by intercepting all data on IPC channels. This 
permits test harnesses to inspect or modify the content of any communication between cooperating processes 
and control the behaviour of the processes involved by injecting or altering the data communicated. This 
includes the IPC mechanisms of anonymous Pipes, FIFOs (named pipes), files, standard IO, Message 
Queues, Sockets and Shared memory. 

To facilitate network programming, ICE provides a fully configurable dispatch queue-based 
multithreaded TCP server that can be invoked by user programs or test harnesses. This TCP server permits 
the definition of a callback function for processing requests made to the server. This reduces the barriers for 
students to developing network software by avoiding the chicken and the egg problem of requiring either a 
TCP server or TCP client to be working in order to test the other. User exercises can focus on developing 
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socket client programs since ICE provides a reliable TCP server to test against. Test harnesses can also 
perform detailed white-box testing of TCP client user programs as test harnesses can configure the server 
callback function to handle and assess client requests, while simultaneously controlling the behaviour of the 
client via standard input injection and assessing its reactions to server responses. 


3. TEACHING WITH ICE 

ICE has been used in the teaching of advanced programming classes covering systems programming and 
distributed systems programming at Griffith University. The laboratory environment used in these classes 
includes both MSWindows and Cygwin/Linux operating systems. Specifically, ICE has been used to 
teach/assess the following set of concepts: language constructs for C/C++ (optional inline assembly); 
dynamic data structures; algorithms; network programming; file system and device IO; multiprocessing and 
inter-process communication and multithreading and synchronization. Students are permitted to do the 
practical lab work using the development tools of their choice. 

ICE provides a means for students to thoroughly test their programs to ensure they correctly meet 
exercise requirements. Some students choose to initially implement their solutions using native tools such as 
gcc/make and MS Visual Studio and then use ICE’s group learning features to check the performance of their 
solutions against the rest of the class. ICE is also used in assessed laboratories to automatically grade student 
work. ICE also supports plagiarism/similarity detection and provides comparative performance statistics. 

Automatic plagiarism detection relies on evaluating some measure of similarity between student 
programs (Sheneamer 2016; Roy 2009; Buric 2013). While a variety of different techniques are used the 
most common approach is to measure the similarity between sequences of tokens. These sequences are 
formed by removing all comments, whitespace and identifier names from software programs. The remaining 
code statements are replaced with tokens and the resulting sequences are directly compared. In ICE the 
Levenshtein Distance (Levenshtein 1966) is used to compare the sequences. In addition to comparing 
program source code, ICE also measures the similarity of the object code of the compiled files. These 
measures are reported to students so that they can see how original their solution to any given exercise is and 
to discourage them from simply cloning other students’ solutions. 

The code similarity measure is just one element of information that ICE provides to users regar ding their 
performance. Each time an exercise is compiled, the test-harness-generated score and normalized program 
execution time is sent to, and collated by, the ICE server. The ICE server then responds by sending the ICE 
client updated statistics about the performance of the entire student group. 

Users can access this information from the “Stats” tab of the main screen as is depicted in Fig. 4. The top 
half of the screen presents information about the individual student’s performance for the selected exercise 
against the rest of the student group. The bottom half gives overall information about the performance of the 
individual student relative to the rest of the student group over all of the set exercises. The top row of bar and 
pie charts presents the student’s and the groups’ test harness scores for the selected exercise. The second row 
of bar charts presents user’s code similarity measure or “uniqueness” and also the execution time relative to 
the rest of the student group. The third row of bar and pie charts presents the total number of successfully 
completed exercises for the individual compared to the rest of the group. The last row presents the total score 
achieved by the individual over all exercises attempted compared to the rest of the group. In each bar chart 
the horizontal line represents the score of the user whereas the bars are the sorted individual scores of the rest 
of the group. Permitting students to be immediately aware of their standing in a group of their peers 
encourages a sense of competitiveness in some and an awareness of a need for improvement in others. Both 
of these responses can work towards improving student learning outcomes. 

The student group was surveyed to ascertain their perception of the usefulness of the ICE environment, 
the performance statistics data and other features. The results are given in table 1. A total of 75% of 
respondents found the comparative performance information useful in their learning with the remaining 25% 
being uncommitted. Students in general found a number of ICE features very useful, such as the simplified 
build environment/user interface. A majority of students found the ability of ICE to perform both WIN32 and 
UNIX builds and testing of their code rather than needing to use multiple tools on different operating systems 
useful. In general, on a scale of 1 to 5, where 1 indicated that ICE was not useful and 5 was that it was 
potentially very useful, ICE was awarded an average score of 4, with 50% of students awarding ICE a score 
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of 5. Students identified a number of teething issues and/or feature enhancements that they wanted. These 
include more a powerful text editor with infinite undo, more detailed reporting of compiler errors, and an 
ability to download their past exercise solutions from the server without having to access the copies that are 
automatically saved locally. 


File Edit Search View 


tl X 1) ^ Login: |k Pass: 



Line 1 


Figure 4. View of Statistical Data for an Individual Exercise 


Table 1. ICE User Survey Responses 


ICE Feature 

Useful 

Unsure 

No Useful 

Did not use 

Performance Statistics 

75 % 

25% 



Both Win32 & Unix 

62 % 


12% 

25% 

Simple User Interface 

75 % 


25% 


Automatic Testing 

50 % 

38% 

12% 



From a teaching perspective, ICE encourages students to complete their laboratory exercises as it gives 
students visibility as to their individual performance relative to the rest of the student group. From the 
collated student scores, it gives the teacher an instantaneous view into the performance of the entire student 
body and the performance of individual students within it. ICE discourages students from plagiarism and 
provides immediate feedback to students regarding the correctness of their solutions. It facilitates the grading 
of student work. It also simplifies issues related to needing to support multiple development environments for 
laboratory work. Most importantly it permits a class of advanced programming techniques to be assigned and 
assessed that is not possible with other software tools. 


4. CONCLUSION 

ICE is a specialized software tool for facilitating teaching and learning of advanced programming concepts. 
While it shares some of the objectives of other tools, it is unique in its capabilities for simplifying the testing 
of platform specific code with different dependencies by simulating each separate platform and for providing 
support for testing advanced programming features such as interprocess communication, device IO, 
multiprocessing and network programming, etc. In addition to functional testing, ICE also automatically 
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evaluates execution time and tests for plagiarism. The results of each of these tests are immediately available 
to the user for comparison to those obtained by others in the same learning group. The majority of students 
surveyed found ICE to be useful in their learning and staff likewise finds it useful for supporting teaching. 
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ABSTRACT 

"'iMOOClOl: Mastering American e-Learning” is a Coursera-based, free, massive online course aimed at preparing 
non-U.S. students to succeed in regular, for-credit, online classes in American universities. The course is also intended to 
help foreign-born professionals integrate into virtual work environments in U.S.-based companies. The development of 
the course is informed by two decades of practical experience of one U.S. public university in the field of international 
distance learning. The course is designed as a virtual laboratory of culture shock (Chukhlomin, 2010. 2016) where 
students are exposed to unfamiliar virtual environments and learn how to understand and adapt to the American online 
culture, the academic system, and professional contexts. The course had initially been developed as a small-scale, 
for-credit, online course; in 2014-2015 we obtained two Innovative Instructional Technology Grants to transform it into a 
MOOC format. Here we report on the design principles and the first results. 


KEYWORDS 

Virtual Culture Shock, Acculturation, Barriers, MOOC 


1. INTRODUCTION 

Research has shown that non-native online learners and working professionals face significant barriers when 
they immerse into virtual learning and work environments in another country (Zhang & Kenny, 2010; Liu et 
al, 2010). One way to address this problem is to use transition, or bridging, courses that can help non-natives 
overcome barriers and succeed in the new virtual environments (Peelo & Luxon, 2007; Evans & Northcott, 
1999). According to Chukhlomin, Deshpande and Chandra (2013), in the context of international online 
learning in the U.S. a transition course is one that helps international online students better understand and 
become familiar with American culture, the academic system, and the ways of doing things. This includes 
learning about the use of specific learning management systems, refining critically important skills 
(communications skills, research skills, cross-cultural skills), practicing some of the widely used in the U.S. 
educational technologies (electronic libraries, videoconferencing, e-portfolios), modelling expected behaviors 
(time management, open discussions and debates, teamwork), and adjusting to student-centered pedagogies. 
By taking a transition course, a non-native learner can overcome or significantly lower the barriers and 
become better prepared for studying subsequent subject matter in an American university or taking a virtual 
job with an American employer. 


2. VIRTUAL CULTURE SHOCK AND VIRTUAL ACCULTURATION IN 
TRANSITION COURSES 

A guiding conceptual framework for understanding barriers and developing alleviating sUategies in transition 
courses is provided by the theory of culture shock introduced by Oberg (1960) and refined by Ward et al 
(2001). Building on the Ward’s ABC model of culture shock (2001, p. 270-272), virtual culture shock is 
further defined by Chukhlomin (2010, 2016) as a situation where a non-native learner (“virtual sojourner”) is 
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suddenly exposed to a completely unfamiliar virtual setting and is wholly overwhelmed by it (this is the “A”, 
the affective component of the model). Two other components (the “B” and the “C”) are behavioral and 
cognitive; the cognitive component is the one that enables the individual to reflect on the situation, 
understand its root causes and consequences, and to take action. The behavioral component deals with the 
practicalities of instrumental adjustment helping the virtual sojourner adapt to the new online environment 
and learn how to navigate and even thrive in it. The components of the model are interconnected and support 
each other. Before the behavioral component is engaged, the affective signal should be received and correctly 
interpreted by the learner as the need to adapt. 

Virtual acculturation, the intended outcome of the transition, is to be achieved by walking the individual 
through the following steps: a) exposure to a situation of a virtual culture shock; b) debriefing and analysis; 
c) competence building. In the context of a transition course, virtual acculturation is the major learning goal. 
Two subordinate learning objectives are the following: a) developing in virtual sojourners the ability to 
recognize cultural, linguistic, technological, and other barriers that inhibit learning in the foreign online 
environment; b) building competencies enabling them to overcome the barriers and effectively function in the 
new environment. 


3. THE IMOOC PROJECT 

For more than two decades, a group of the University’s faculty has been engaged in delivering transition 
courses for remote cohorts of non-U.S. business students and working professionals utilizing various 
scaffolding tools and techniques, such as facilitated group studies, bilingual and bicultural instruction 
(Chukhlomin et al, 2013). Initially, transition courses were only offered as small-scale, for-credit, online 
courses. The arrival of MOOCs has created new opportunities for remedial education (Bonk el al, 2015). In 
2014, we obtained support from SUNY IITG to fund a project (now known as “the iMOOC Project”) aimed 
at converting a series of transition courses into a MOOC format. In 2015, we piloted a facilitated MOOC; in 
2016, we re-designed it and launched a permanent, on-demand version of that course on Coursera 
(https://www.coursera.org/e-learning). 


3.1 Course Design 


“iMOOClOl: Mastering American e-Learning” is a facilitated version of the massive online course that is 
aimed at preparing international online learners to successfully transition to virtual learning and work 
environments in U.S. universities (Figure 1). 


Introduction 

What would you do if you 
were the only non-U.S. 
student in an all-American 
online classroom? 

O Alex: How docs it work, 3 m j n 
actually? 

^ The problem that we are 
(►) trying to solve in this 3 min 


Practice Quiz: 

(S' Week 1 4 questions 

Self-assessment 


competency-based 
approach to overcome 


What is uniquely American 
► in U.S.-based online 5 min 


Practice Quiz: 

Week 1 Lesson 2 

practice quiz: 12 questions 

competencies 



Figure 1. Screenshot from iMOOClOl “Mastering American e-Leaming” 

The target audience of the course includes the following groups of learners: 

1. Remotely located, international online learners - either in organized cohorts or as individuals. 

2. International students located outside of the U.S. and enrolled in dual degree programs established by 
their home institutions with U.S. universities. 
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3. International students preparing for departure to a U.S. educational institution. 

4. Beginning international students that are already in the U.S. and looking for additional support. 

5. Recent immigrants to the U.S. or those who are considering employment with U.S.-based companies. 
According to Adeniran el al (2008), a good starting point for designing a transitional program is to create 

a comprehensive list of barriers and to identify competencies that are needed to overcome the barriers. For 
the iMOOC project, we used an original taxonomy of barriers, alleviating strategies, and required 
competencies which included 10 types of barriers and provided 69 diagnostic statements for self-assessment 
(Chukhlomin, 2016). The taxonomy served as a guiding pedagogical framework for the backward design 
(Wiggins and McTighe, 2005) of the course. In the final version, the course included six learning modules: 
Module 1: Introduction, Module 2: Technology, Module 3: Language and Culture, Module 4: 
Communicating Ideas, Module 5: The U.S. Academic System, and Module 6: Professional Contexts. 
Overall, for this course we created 85 original videos and animations, 45 content guides, 27 self-assessments, 
a range of moderated activities (discussions), pre- and post-course comprehensive self-assessments, and a 
concluding final paper assignment. We also filmed interviews with Lebanese students taking regular, for- 
credit, online courses at a U.S. university. 

3.2 The Pilot 

The first facilitated session of the course was conducted on Coursera in March-May 2015 as a tuition-free, 
certificate course. To attract students, the course team relied on Coursera marketing department, but also 
reached out to the U.S. Department of State and the University’s Global Center. In addition to open 
enrollment, there were supervised groups of students in New Paltz, NY and the University’s programs in 
Greece and Panama. Additionally, there were several student groups in Indonesia observed by local 
EducationUSA staff. The course length was 6 weeks, with additional optional modules. During that time 6 
faculty members facilitated course discussions. 

According to the Coursera, 4,887 students signed up for the course. The learners followed a typical for 
Coursera (Glance and Barrett, 2014), “L-shaped”, exponential pattern of participation (Figure 2). 

. ’ 38,872 3,167 1,511 

O Iecture3wewed ET3 exerases suDmjtted ^ forum oo3ts made 


Lecture Activity 



Figure 2. Content activity in iMOOClOl 

Overall, 3,071 students ever visited it, 1,840 watched a lecture, 628 browsed a discussion, and 423 
submitted an exercise. There were 17,368 total streaming video views and 24,020 video downloads; 430 
unique participants did 3,186 quiz submissions. 278 participants submitted 1,188 unique discussion posts in 
388 threads; those posts received in total 6,902 views. 48 participants chose to get a course certificate and 
submitted their final projects for peer-review; they also provided in total 164 peer-review evaluations; as a 
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result, 40 course certificates were granted. In terms of geography, the learners represented 145 countries, 
with 21% from the U.S., 15% - China, 5% - India, 4% - Mexico, 3% - Russia, 3% - Indonesia, 3% - Panama, 
2% - Brazil, 2%. 

In the beginning of the course, the learners were asked to clarify what topics they perceived to be the 
most interesting for them in the course; 448 students responded to this question. The responses indicated that 
the course indeed captured attention of the intended target population. “How a non-U.S. student residing 
outside of the U.S. can gain knowledge and develop skills to succeed in online classes offered by a U.S. 
college or university” was selected by 45% of all respondents; “How a working professional residing outside 
of the U.S. can develop skills that are useful in working with or for a U.S. organization” - 24%; “How a 
U.S.-based working professional can develop a better understanding of virtual environments in a 
cross-cultural perspective” - 13%; “How an international (non-U.S.) student residing in the U.S.-based 
campus can develop skills for online studies” - 12%,“None of the above” - 6%. 


4. EVALUATION STRATEGIES 

To evaluate the first pilot results, the course development team used the following methods and tools: 

• Built-in self-evaluations in learning modules; 

• The final paper; 

• Pre- and post-course comprehensive self-assessment; 

• Content analysis of discussions; 

• Post-course survey assessment. 

4.1 Post-course Survey Assessment 

The post-course survey questionnaire was based on a survey instrument developed by Zaharias and 
Poulymenakou (2009). The survey instrument focuses on specific constructs, such as content, visual design, 
navigation, accessibility, interactivity, self-assessment and motivation to learn all of which were found to be 
critical in examining the overall learner engagement and experience. The survey included both Likert scale 
questions and open-ended questions. The total number of questions in the survey instrument were 52. Some 
sample questions open ended questions were as follows: 

• Describe whether the iMOOC helped you better understand the American cultural and academic 
systems. If so, how? 

• Describe the benefits to have emerged for you by taking the iMOOC. 

• Describe some of the challenges you faced in successfully completing the activities in the course. 

• What would you like to see done differently in the iMOOC? 

The participants of the course were asked to report whether the iMOOC helped them become familiar 
with and better understand American culture and the academic system. The results indicated that close to 50 
% of the respondents agreed and over 35% of the respondents strongly agreed with this statement while only 
2.6% disagreed. The participants were asked if they found the iMOOC enjoyable. The results indicated that 
49.4% agreed and 36.4% strongly agreed while only 1.3% disagreed. Additionally, 48.1% agreed and 32.5% 
strongly agreed with the statement that the iMOOC provides instruction and training that matched the 
learners’ experience. Furthermore, close to 35 % of the respondents agreed and over 55% of the respondents 
strongly agreed to the view that various resources (such as video-based lectures and interviews, weblinks, 
case studies) present within the iMOOC were valuable for supporting their learning. 

Discussions formed a major interactive component of the iMOOC. The results indicated that 44.2 % of 
the respondents agreed and over 37% of the respondents strongly agreed to the view that the iMOOC 
provided opportunities and support for learning through interaction with others specifically via discussions 
while only 1.3% disagreed. Respondents of the iMOOC were also asked to report whether the iMOOC 
provided learners with control of their learning activities and helped them take ownership of learning. The 
results indicated that close to 52 % of the respondents agreed and over 35% of the respondents strongly 
agreed to this view while only 2.6% disagreed. The learners were also asked to describe what learning 
resources within the iMOOC helped them better understand the American culture and academic systems. 
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Qualitative analysis of the data (Deshpande & Chukhlomin, 2015) revealed that the majority of the learners 
indicated that video-based lectures specifically helped them understand American culture and academic 
system. 


5. FUTURE WORK 

Our results demonstrate that students who persisted in the course were satisfied with the learning offered by 
the iMOOC. A vast majority of the survey respondents indicated that activities and resources within the 
iMOOC helped them better understand the American academic and cultural system. However, further efforts 
are required to determine the impact of the iMOOC on students’ ability to overcome barriers and develop 
necessary competencies. 
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ABSTRACT 

English language proficiency is an increasingly vital skill for employment in Saudi Arabia. However, compulsory 
English as a foreign language courses at all levels of education have only produced inadequate results. One of the issues 
consistently raised as a barrier to effective EFL learning is the use of passive learning pedagogies and the lack of 
self-directed or deep learning in the Saudi EFL curricula. Increasingly, educators globally are turning towards technology 
to engage their students in more effective learning and they acknowledge that, in order for technologically-enhanced 
pedagogies to be effective, it is important for them to be congruent with the ways in which learners actually use their 
devices. However, current research has failed to explore how students use technologies to learn outside of the classroom. 
Therefore, this research aims to identify how university students at Saudi University taking a compulsory preparatory 
EFL course use their mobile devices to practice and learn English outside of the classroom. To do this, three phases of 
data collection are used, one quantitative and two qualitative. Phase one is a quantitative online survey measuring what 
mobile devices students use for learning, and what English language learning activities they engage in with those devices. 
Phase two involves the collection of 25 multi-modal journals over a two-week period using the WhatsApp messaging 
platform to collect text and pictures of participants using their devices to learn in real time. Phase three uses both a focus 
group interview with the 25 journal participants and individual interviews with each in order to explore the recorded 
activities in more depth and relate them to models of self-directed learning. Thus, this research aims to collect rich data 
from multiple perspectives on how, where and why university students use mobile devices for learning outside the 
classroom in Saudi Arabia, in order to develop effective in-classroom pedagogies. 


KEYWORDS 

Self -Directed Learning, Mobile Learning, Outside of the Classroom 


1. INTRODUCTION 

This research explores the intersection between three dynamic and contemporary areas of English as a 
Foreign Language (EFL) research: Mobile Learning (m-learning), Self-Directed Learning (SDL) and 
Learning outside the Classroom. Its primary aim is to explore how university level students use mobile 
devices to learn and practice English as a Foreign Language outside of the classroom in Saudi Arabia, how 
they perceive these activities and whether these activities can be accurately described as Self-Directed 
Learning according to current models of SDL. The results of this research are expected to contribute to both 
the development of new pedagogies in Saudi Arabia and new methodological and empirical knowledge to the 
field of EFL teaching and learning through its innovative use of multi-modal journals for data collection. 

1.1 Background to Study 

Despite being a compulsory part of intermediate, secondary and higher education in Saudi Arabia, EFL 
proficiency among young people in Saudi Arabia remains low, and courses are producing consistently 
inadequate results (Alrabai 2016; Al-Nasser 2015; Khan 2011; Alrashidi & Phan 2015). English language 
proficiency has become vital to employability in Saudi Arabia, as a result of the economy’s reliance on 
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Anglophone foreign oil companies (Al-Abed A1 Haq & Smadi 1996; Alhaisoni 2013; Golum Faruk 2014). 
English language education, therefore, is a priority of the National Transformation Program, Vision 2030 
(Vision 2030 2016). 

One of the issues that is commonly raised by researchers and teachers in Saudi Arabia as a barrier to 
successful EFL learning is the over-reliance on passive learning strategies in Saudi Arabian classrooms, 
strategies that are teacher-led and only encourage surface learning (Al-Seghayer 2015; Al-Seghayer 2014). 
There has, therefore, been a push within Saudi Arabia to formally encourage pedagogical strategies that will 
encourage self-directed and deep learning, including the use of mobile and e-learning. 

Mobile learning, in particular, is seen as an effective and engaging method of enhancing EFL teaching 
and learning and fostering SDL. It has been noted that mobile learning can be particularly effective for EFL 
teaching and learning in contexts where students have limited natural opportunities to use English outside of 
the classroom (Richards 2015). However, researchers have highlighted that, in order for a mobile learning 
initiative to be successful, it has to be congruent with the ways in which students already use their mobile 
devices (Lai & Gu 2011). Furthermore, as mobile learning is not currently formally embraced by the Saudi 
Arabian Ministry for Education, and is therefore not a part of the formal EFL curriculum, it is only possible 
to explore how students are using their devices for learning outside of the classroom. 

1.2 Study Rationale 

English is the lingua franca of business in Saudi Arabia as a result of the predominance of the foreign oil 
industry in the Saudi Arabian economy. At this time, Saudi Arabia is undergoing a national transformation 
which explicitly aims to increase the size of the private sector in the country and increase the number of 
Saudi Arabian nationals employed by the private sector (Vision 2030 2016). This includes plans to provide 
“citizens with the knowledge and skills necessary to meet the needs of the future labor market” (Vision 2030 
2016, p.57). A core necessary skill is English language proficiency. As a result, it is a key aim of the Saudi 
Arabian government to improve EFL education at all levels. 

Notably, however, almost the entire body of current research has explored EFL teaching and learning 
only from the personal experiences of EFL teachers or general cultural commentators (Alrabai 2016; 
Al-Nasser 2015; Khan 2011; Alrashidi & Phan 2015). Very little has explored the perspectives of students, or 
considered their current attempts to learn or practice English outside of the classroom. 

These conversations around EFL learning and the importance of English in Saudi Arabia have occurred at 
the same time as the worldwide digital revolution. Saudi Arabia is becoming a nation connected by mobile 
devices (IPSOS, 2012). Social media and messaging application usage is rising across the Arab world, and 
most users interact with social media on their smartphones (Dubai School of Government, 2015). In Saudi 
Arabia, 75% of smartphone owners use them to access social networks, while the social messaging network 
WhatsApp is used by 41% of social media users (IPSOS, 2012; Dubai School of Government, 2015). Initial 
studies on the use of mobile learning in Saudi Arabian classrooms have been positive and are likely to 
increase (Abu-al-aish & Love 2013; Al-Fahad 2009; Chanchary & Islam 2011; Alsaleem 2013). 

Understanding how students are currently using their mobile devices for learning and practicing English 
as a Foreign Language is important for the successful development of effective mobile learning pedagogies 
in the classroom (Lai & Gu 2011). At the same time, however, it is equally important to understand how far 
their current practices align with models of SDL in order to understand the effectiveness of these activities. 


2. LITERATURE REVIEW 

This study takes place in Saudi Arabia, which has a unique political, cultural and historical context. Saudi 
Arabia was founded in 1932 but did not encourage the learning of EFL for many decades as a result of the 
rigid religious culture that rejected the Western world, including their languages. The first EFL curriculum 
was developed in 1980 and is still controlled by the Ministry of Education, which ensures that EFL teaching 
meets the moral, cultural and religious concerns of Saudi Arabia (Al-Seghayer 2014). Among the strategies 
implemented by the Ministry of Education to improve EFL outcomes at all levels of education is the use of 
information technologies in the classroom (Golum Faruk 2014). Implementation has, however, been 
inconsistent and hampered by lack of teacher training and lack of access to ICT resources (A1 Gamdi 
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& Samarji 2016). Therefore, it is learning outside the classroom that has been primarily transformed by 
advances in information technologies. (Rashid & Asghar 2016; Candy 2004). Indeed studies in Hong Kong 
and Australia have demonstrated that significant numbers of students are using their mobile and internet 
devices for SDL outside of the classroom to regulate and facilitate their learning both inside and outside the 
classroom (Lai & Gu 2011; Farley et al. 2015). This is particularly the case in language learning, the purpose 
of which is to use the target language in natural contexts. Indeed, it is widely acknowledged that solely 
classroom-based learning of languages is likely to be ineffective (Benson & Reinders 2011). Many students 
globally, however, have found ways to use the internet to use and practice English in natural contexts 
(Benson & Reinders 2011; Kuure 2011; Beatty 2015; Richards 2015). Notably, however, students do not 
always conceptualize their use of English online as EFL learning, a phenomenon which highlights the 
complexities of learning outside the classroom (Benson & Reinders 2011; Kuure 2011). 

Mobile learning is strongly related to learning outside the classroom. Mobile learning refers exclusively 
to learning that occurs while the learner is not in a classroom or fixed location, using a mobile device 
(Kukulska-Hulme 2009; Kukulska-Hulme & Shield 2008). It is therefore dislodged from formal locations or 
specific times, flexible and self-regulated. However, it remains almost entirely unstudied within a Saudi 
Arabian, or Gulf, context. 

If students in Saudi Arabia are using mobile devices outside of the classroom for EFL learning then a 
question arises concerning whether these activities can be conceptualized as effective Self-Directed Learning. 
Self-Directed Learning (SDL) is defined as a “process in which individuals take the initiative, with or 
without the help of others, in diagnosing their learning needs, formulating learning goals, identifying human 
and material resources or learning, choosing and implementing and evaluating learning outcomes” (Knowles 
1975, p.18). The most commonly used model of SDL is Garrison’s (1997), which conceptualized three 
necessary dimensions for effective SDL: self-monitoring (responsibility), motivation (entering/task) and 
self-management (control) (Garrison 1997, p.22). SDL is therefore conceptualized as a complex interplay 
between cognitive, metacognitive and behavioral processes where multiple dimensions are necessary for 
effective and useful learning (Navarro & Thornton 2011; Bonyadi et al. 2012; Loyens et al. 2008) 


3. RESEARCH METHODS 

This primary aim of this research project is to identify how EFL students in higher education are currently 
using mobile devices for EFL learning outside of the classroom. To meet this aim, it will answer the 
following research questions: 

1. In what specific activities do students engage to learn EFL on their mobile devices outside of the 
classroom? 

2. How do the activities identified by the participants relate to models of SDL? 

3. What potential drivers of and challenges to using mobile devices outside of the classroom for EFL 
learning are identified by participants? 

4. How do the activities identified by the participants as being used to learn English outside of the classroom 
relate to the participants’ in-class EFL experiences? 

To answer these questions, this research uses a mixed methods design with three phases of data 
collection, one quantitative and two qualitative. Data will be collected from students in their preparatory year 
at Saudi University taking the two compulsory English language modules. The overall cohort is 
approximately 4,000 prepatory year students. Only female students will be eligible for this research because 
of the cultural norms of Saudi Arabia which prevent personal contact with male participants. 

Phase one of the research will be a quantitative online survey, which will aim for 450 respondents. The 
survey consists of 45 questions, the majority of which use a 7-point Likert scale to measure the frequency 
with which participants use different mobile devices (such as smart phones, tablets and e-readers) to engage 
in specific learning activities such as watching videos in English, using social media in English, reading 
English articles and so on. 

Phase two will involve 25 participants, drawn from the survey respondents, who will complete 
multi-modal journals using the WhatsApp messaging platform for a period of two weeks, documenting their 
use of mobile devices for English learning in real time using pictures and text. Participants will be asked to 
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screenshot or photograph their activity and record it in a WhatsApp message along with a small piece of text 
explaining the activity and describing the location of the activity. Multi-modal journaling has been chosen 
here instead of online journaling because this phase of research will require participants to capture and 
upload their activities as they are doing them, taking advantage of the flexibility and mobility of mobile 
devices, rather than waiting until they are at a desktop computer to write formal journal entries (Gourlay et al. 
2015; Gourlay 2010). The aim of the multi-modal journals is to collect rich, thick qualitative data about the 
precise EFL learning activities that participants engage in using mobile devices, how often each activity 
occurs and what these activities mean to the participants. 

In phase three, these 25 participants will be invited to participate in both a focus group interview and 
individual semi-structured interviews. These interviews will focus on the contents of the multi-modal 
journals and will aim to relate the activities recorded in the journals to models of SDL in order to explore 
their effectiveness and explore the meaning and purpose of the activities in more depth. The focus group 
interview will be used as the basis for the development of interview schedules for the individual interviews. 

The data in each phase will be analyzed before the next phase begins. At the end of the data collection 
phases, all the data will be collated and analyzed as a whole in order to comprehensively answer the research 
questions and produce useful and appropriate recommendations for a Saudi Ar abian context. 


4. CONCLUSION 

This research both contributes to ongoing theoretical conversations about SDL, mobile learning and learning 
outside of the classroom, and to policy priorities within Saudi Arabia. It is also methodologically innovative, 
both in its focus on students’ learning activities outside of the classroom and in the use of multi-modal 
journals, which have not been used before. It is anticipated that this project will produce data that will 
contribute to the development of new and innovative ELL pedagogies in Saudi Ar abia at a time of national 
transformation when English learning is a national priority. At the same time, it is anticipated that the results 
of this research will contribute valuable methodological, empirical and theoretical findings to the field of 
EFL teaching and learning globally, across multiple areas of research. 
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ABSTRACT 

In this study, the role-playing situation and the system requirement list are adopted into project-based learning classes to 
develop web applications. In the classes, the third-year undergraduate project managers communicate with the client of 
the project rolled by teachers on the Web bulletin board. These are expected to act as scaffolds to promote 
communication between the project manager and the client. The project management class without these scaffolds was 
held in 2014 and 2016. The results show that these scaffolds seem to have worked effectively to some extent. 
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1. INTRODUCTION 

Project-based learning is effective because it makes learning deeper and builds social skills. PBL is 
performed in various tasks and environments. The project management class for the third-year 
undergraduates has been held in our university's information department (Sasaki, et al., 2014). After students 
have been taught about project management, they are given opportunities to practice what they have learned 
about project management. The first-year undergraduates work on team problem solving in the PBL class. 
The third-year undergraduates act as a project manager (PM) for the first-year undergraduates' team. 
Although the aims of these classes are different, both of them could build their social skills and improve their 
performance (Sasaki et al., 2014). 

However, about project management, we found some problems. One of the main problems is that the 
third-year undergraduate PMs (3yPMs) tend to decide specifications of the Web system without hearing 
opinions from the client. In this case, the teacher acts as the client of the Web system development project. 
Some of these 3yPMs have a tendency to design the system based only on their own thoughts and seem to 
disregard the client’s needs. Moreover, at times they have been known to just ask the teacher to provide them 
with the answer. Even in such cases the 3yPMs work on complex activities such as system programming, and 
they appear to be sincere and serious about their work. Woods has noted that unsuccessful problem solvers 
tend to spend most of their time doing something, whereas successful problem solvers spend most of their 
time deciding what to do (Woods, 1994). It seems that understanding the process of problem solving is very 
important, particularly for people in project management, as it is their responsibility to determine what must 
be done to achieve success on a project. One of the effects of the role-playing is that it puts added values like 
creativity and a student-driven approach into traditional PBL, which emphasizes problem-solving skills 
(Chan, 2012). The 3yPM will propose their own solution to the client and get opinions about it. In this 
process, creativity and a student-driven approach are necessary for the 3yPM. In addition, it is expected that 
the requirements and specification list might help the 3yPM to organize the proposal systematically. 

This study aims to promote communication between the 3yPMs and the client so that 3yPMs exchange 
appropriate information with the client and understand the needs of the client. To address this issue, the 
role-playing situation and the requirement and specification list are adopted when the 3yPMs communicate 
with the client on the Web bulletin board. These are expected to act as scaffolds for effective communication. 
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2. COMMUNICATION ON THE WEB BULLETIN BOARD 

On the conception and initiation step of the project management, the 3yPM defines the specification of the 
Web system based on the clients' order document. Because the order document is just a rough description, it 
is necessary to hear the opinion from the client about the concrete function. The 3yPMs make the 
requirement and specification list. The requirement and specification list includes “item ID”, “requirements 
and specifications proposed by PM”, “comments from client” and “remarks”. The 3yPM will be able to 
realize requests and specifications concretely by making the requirement specification list. It becomes clear 
on this occasion if there is an uncertain part. The 3yPM posts a message with the requirement specification 
list attached, and writes questions to the client if necessary. 

The 3yPM writes a message as playing the role of the employee of the software manufacturer. The 
teacher playing the client write responses from the client. This role-playing is expected to lead the 3yPMs to 
deeper understanding of the requests and the specification. The details of the role-play are as follows. 

Situation setting: A software manufacturer receives an order for the development of a Web system 
from the secretary division of a university. 

Roles: The 3yPM is an employee of the software manufacturer, and a teacher is a client who 
represents the secretary division or a laboratory of the university. 

The work procedures of the 3yPMs are as follows. 

Receive the order document: The order documents for Web system are given to the 3yPMs. Based 
on the explanation of what the goals of the system should be, the 3yPM makes the requirement and 
specification list. 

Sending the message on the client: The 3yPM posts the message on the bulletin board with the 
requirement and specification list attached. 

Communication on the bulletin board: The 3yPM receives comments from the client and iteratively 
adjusts, corrects, and updates the requirement and specification list. The work in this procedure is 
carried out on the bulletin board of the learning management system (LMS). 


3. CLASS PRACTICE 

These scaffolds (i.e., use of the requirement and specification list and the communication on the bulletin 
board) were used in the project management classes in 2014 and 2016. A project management class without 
these scaffolds was held in 2013. The 3yPMs write the project-file as the final outcome on the conception 
and initiation step of the project management. To examine the effect of these scaffolds, the descriptions of 
“the true needs" in the project-file are compared. There are 7 subjects in 2014 and 2 subjects in 2016. 


4. RESULTS OF THE PRACTICE 

The number of messages posted to the bulletin board is shown in Table 1. Half of these messages are replies 
from the client. Table 2 shows the number of project-files which mention of the client's need in "the true 
needs." Some of the 3yPMs wrote only the functions of the Web system or the title of the order document in 
the project-file. Figure 1 shows examples of the communication on the bulletin board. One of the 3yPM sent 
a message with the requirements and specification list attached, and received comments from the teacher. 
The 3yPM played the role of the employee of the software manufacturer and the teacher played the role of 
the client who represents the student support section of the university. 

In 2014, a questionnaire survey was administered after the course. The result of the survey shows that 
60% of the respondents felt the role-playing was useful. From the place provided for giving a free-form 
response, we received these opinions: 

I think the role setting is effective for learning project management. 

I had a good experience and I could exchange emails in a form similar to what is used generally in 
society. 

I felt that the role setting was only perfunctory and was made use of only in writing formal 
documents. 
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Table 1. The Number of Postings to a Bulletin Board 


3yPM ID 

Number of messages 

2014 

2016 

1 

4 

4 

2 

4 

4 

3 

4 

/ 

4 

6 

5 

6 

6 

6 

7 

6 


Table 2. The Number of Project-Files Which Mention About the Client's Need in "The True Needs" 


2013 

2014 

2016 

4 out of 8 

6 out of 7 

2 out of 2 


Subject: Sending requirement specification lists 
Dear Sasamoto, 

I modified some items of the requirement and 
specification lists according to your designation. 

In addition, I added the brief description of the Web 
page design and the reservation information edit 
function for teachers. 

Please feel free to contact me with any concerns or 
questions that you may have. 

Regards, 

M. Hata 

Project Manager, QD Software Service Co. Ltd. 
[Attachments: reqt_specjist.xlsx] 


Subject: Re:Sending requirement specification lists 
Dear Hata, 

Thank you for sending the details of the Web design 
and reservation information edit function. 

Is it possible to indicate vacant facilities on the 
facilities lists page and add reservation function on 
the same page? 

In addition, we would like to add, delete and edit the 
facilities data. 

Are these functions included? If not, please add 
them. 

Regards, 

S. Sasamoto 

Team Leader, Student Support Section, 

TK University 


Figure 1. Example of the Communication on the Bulletin Board, the 3ypm Sent a Message on the Client, and the Client 

Replied to It 


5. DISCUSSION 

From results shown in table 2, it seems that these scaffolds have some positive effect. It seems that the 
client's needs became clear by making the requirement and specification list. However, the effect of the 
role-playing is not clear. From results of the survey, it seems that students thought they managed the 
comments and the requests from their clients appropriately and found the role-playing useful for setting the 
conditions governing their project. In addition, they seemed to feel useful because they could experience the 
social style of being an employee by playing the PM’s role. In this practice, a student and a teacher played 
the role of an employee of the software manufacture and the client of the project. The Japanese students tend 
to feel a hesitation in talking to teachers. Because the student played the role and wrote messages in formal 
business style, it seems that they could communicate with teachers more easily. On the other hand, it seems 
that the negative responses came from a perception that the role-playing was unnecessary and only 
perfunctory. Overall, these scaffolds seem to have worked effectively to some extent. 
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6. CONCLUSION 

In this study, the role-playing situation and the requirement and specification list are adopted to promote 
communication between the 3yPMs and the client so that 3yPMs exchange appropriate information with the 
client and understand the needs of the client. It seems that the 3yPMs could get deeper understanding of the 
intention of client by making the requirement specification list. Though the effect of the role-playing was not 
clear, some students felt a positive effect from the role-playing. 
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ABSTRACT 

The continuous professional development of educators is not only essential to highly maintain their expertise levels and 
ensure that their knowledge is up to scratch, but also to catch up and adopt new pedagogical tools, skills and techniques. 
The advent of the Web 2.0 brought about a plethora of digital tools that teachers have not only struggled to keep track of 
and investigate the array of tools available, but also have been at a loss of how to productively employ and take 
advantage of the benefits each of the different tools has to offer. In this paper we present a number of such tools as part of 
an empirical study to investigate the use of freely available digital tools. This exercise was a spin-off of a project that 
aims to embrace literacy through digital media as it focuses on re-training educators in the use of digitally supported 
methods that employ innovative teaching methods. In our conclusions we draw important guidelines on how to optimize 
the use of such continuous emerging web tools, as well as future work and new potential research directions. 
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1. INTRODUCTION 

The use of innovative teaching methods forms an essential part of the continuous professional development 
(CPD) of teachers especially when one considers the ever-growing challenges and opportunities that become 
available online. Educators require new skills to adapt to the dynamic nature of digital tools and pedagogies 
that offer novel and potentially more effective methodologies. This is even more amplified by the emergence 
of Web 2.0 (O'Reilly, 2005) technologies that changed the way web pages and applications are designed and 
used. According to Davies and Merchant (2009) Web 2.0 is a generation upgrade on the previous static 
World-Wide Web (WWW), where these new dynamic technologies endorse and propagate learner generated 
material together with mechanisms that encourage and support even more interaction between the web users. 
To such extent the authors claim that Web 2.0 technologies have the potential to enrich and transform the 
entire education process and experience as they point out four distinctive characteristics of how learners, 
through Web 2.0 technologies, have the potential to be actively present themselves instead of being passive 
receivers. Learners are also able to modify content itself as well as generate new material that can be 
appended to the content, and at the same time participate in the social activities that such technologies enable. 
These communal practices that Web 2.0 support, empower the learner (Bousaaid, et al., 2015) to actively 
collaborate, further share, and communicate freely with other learners. Such networking capabilities as part 
of each learner’s personal learning environment (Sclater, 2008) have also characterised Web 2.0 technologies 
as they have promoted the development and use of a variety of networking tools (O'Reilly, 2005; Sclater, 
2008) that further support and foster an academic eco-system that learners themselves create and generate 
through online social activities. 

An Erasmus+ project called Breaking Barriers (2007) under the current European Framework Programme 
aimed to take full advantage of such technologies where amongst other objectives it aimed to tackle adult 
literacy by training educators in the use of innovative pedagogies. The project that involved the collaboration 
of eight European further education centres aimed to bring together adult education and digital media by 
exploring how to tackle low-skilled literate adults by engaging them and motivating them through the use of 
digital tools. As the different partners tackled the distinctive difficulties encountered within their own country 
they found a common objective of introducing cutting-edge technologies and tools to tackle such literacy 
challenges across cultures by ensuring their educators were not only aware but fluent in the use of the latest 
web technologies that inspired and facilitated their work. As a spin-off from the project we investigated the 
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capacity building and the professional development of adult educators who participated within the project as 
they were exposed to these new technologies to enhance their digital competence in the teaching of basic 
literacy skills to adult learners. 

The rest of the paper is organized as follows. The next section will delve into the benefits of Web 2.0 
technologies as a number of tools that are later employed as part of the empirical study are highlighted and 
documented. Section 3 will be all about the empirical study itself as the project objectives are revisited, the 
array of tools identified, and the details of the study itself are specified. The following section presents all the 
results collected together with a thorough discussion to evaluate such results, before closing this paper with 
our conclusions and future work. 


2. WEB 2.0 BENEFITS 

The WWW evolved over time from a static document repository where users could access documents 
through their web browsers in a read-only fashion to a read-write environment with dynamic content and 
possibilities for users to contribute, author and participate. This change to the second generation was not as 
simple as it seems as numerous other factors played an important role. The web browsers themselves went 
through a drastic operational evolution to support such a functionality as a struggle between a number of 
browsers was going on to acclaim absolute control of the web users as increase their revenue. On the other 
hand, the World-Wide Web Consortium (W3C) were also working hard to ensure that the required standards 
and protocols are in place. Other technologies that played an important role in the evolution of Web 2.0 was 
the Semantic Web itself together with the support of the extensible Markup Language (XML) that was also 
striving to destabilize its application and effectiveness. Berners-Lee et al. (2001) set out to define how the 
new web generation had meaning and thereby set the trend to develop higher-level applications (Hendler, et 
al., 2002) that could exploit the enhanced capabilities of a smarter web (Frauenfelder, 2001). 

From an educational point of view Web 2.0 technologies and applications have enabled a novel medium 
which teachers and students alike can benefit from as such technologies have “blurred the line between 
producers and consumers of content and has shifted attention from access to information toward access to 
other people” (Brown & Adler, 2008, p. 18). In this way Web 2.0 empowers educators and learners to 
communicate and interact in new and natural ways that was not previously possible over the web thereby 
creating a new educational medium that teachers have to rethink and eventually require re-training. The 
reason behind such reasoning as this novel medium moved the goalposts from students that are receptive to a 
more interactive and creative. Educators are required to push the boundaries on their students to motivate 
them to share, comment, post, create, produce, edit and assess other students’ work and contributions. This 
also helps students gain confidence in themselves as they adopt critical thinking skills as well as useful social 
skills as they interact with peers, educators, knowledge providers and other web users that will assist them 
during the education process and life in general. 

From a practical point of view in reality learners are already making extensive use of Web 2.0 
applications in their life outside the educational arena, so employing tools which they are already accustomed 
to and which they are happy to use is an added advantage to the educational process that facilitates the 
educator’s life. The challenge here is for the educator to select the most suitable Web 2.0 tool to employ that 
appropriately fits with the pedagogy being adopted. Penney (2012) depicts a number of Web 2.0 tools within 
Bloom’s taxonomy to assist educators in their quest to employ the right tool for the right job. Figure 1 shows 
an adaptation of this image with tools that have been employed during the empirical study itself. The same 
idea of looking at the available tools through the perspective of Bloom’s taxonomy brings out the point that 
some tools are applicable to more than one domain and so they can move up and down the taxonomy as need 
to be employed by the educator as long as the tools serves to achieve the intended outcomes as planned 
within the respective learning objectives. Some of these tools as mentioned earlier could already be 
successfully employed by students like for example using Skype, Viber or WhatsApp to communicate 
amongst themselves, Instagram, Diigo, Imgur and Scoop to bookmark and share interesting stuff online, 
Padlet, Prezi and Slideshare to distribute presentations, or Quia, SurveyMonkey and Doodle to set polls and 
appointments. The educator is required to make use of these same tools but for educational goals in a way 
that promotes collaboration amongst learners to achieve a common goal or to generate an engaging learning 
task that requires group work. 
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Evaluate 


Analyze 


Apply 


Understand 
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Figure 1. Adaptation of Digital Bloom's (Penney, 2012) 


3. EMPIRICAL STUDY 

The purpose of this was to investigate the use of digital pedagogies through the use of Web 2.0 tools during a 
CPD course for adult educators to address the main objective of the Breaking Barriers project, namely to 
address adult illiteracy. Participants were required to research a randomly assigned tool from the list provided 
in Table 1 and to complete a given task by using the same tool. The research questions that guided this 
empirical study were: 

i. What is the purpose of employing digital pedagogies? 

ii. What benefits / challenges of employing digital pedagogies? 

iii. How best to employ digital pedagogies in education? 

A group of a hundred adult educators were asked to participate in an online survey out of which a 
convenient sample of 80 participants was formed. They were asked to access and complete an anonymous 
online form in March 2017. The form randomly selected a Web 2.0 tool and each participant was asked to 
thoroughly investigate the particular tool and then answer a series of five questions, namely: 

a. What is the purpose of this particular tool? 

b. How can it be fruitfully applied in class? 

c. Describe how it can be used as part of a pedagogy to teach adult literacy 

d. Identify any challenges encountered 

e. List any recommendations or tips to employ this tool 


4. RESULTS 

All the data submitted by the participants was automatically saved within a spreadsheet which was then 
analyzed to develop a grounded theory by employing the constant comparative method (Strauss & Corbin, 
1990). The authors suggest that this method is suitable to create a theory of how the social world works when 
applied to social units of any size as it connected to that reality that it has been employed to explain. The data 
collected was meticulously coded within the thematic content analysis software called NVivo (2017), that is a 
dedicated software application which performs qualitative data analysis, especially for unstructured and 
non-numeric data. Thematic analysis traditionally involves six sequential and incremental steps (Braun & 
Clarke, 2006), starting with a familiarization of the collected data and generating initial codes, followed by a 
search for themes, reviewing of the themes, defining and naming the themes, and finally employing the 
constant comparative method to generate a theory. 

Following the data entry process of the collected responses within NVivo, three thematic containers or 
nodes reflecting the three research questions that guided this research, and influenced the accumulation of the 
responses around the questions within the online form, namely: Purpose of Web 2.0 tools. Benefits / 
Advantages vs Disadvantages / Challenges, and Recommendations. 

As expected an array of mixed results were reported on the purpose of the different tools as they all serve 
a different purpose with the exception of a couple of tools that perform the same task. However it was 
interesting to notice small clusters in the terminology used by the participants to describe the purpose. These 
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clusters can easily be traced to the different levels of the Bloom’s taxonomy in Figure 1. For example a 
number of tools are used to collaborate or moderate which can easily be associated with the higher order 
thinking skills of creating where participants used words like constructing, making, and producing to describe 
the purpose of the tools they investigated. Similarly tools employed to debate and comment can be associated 
with the evaluation layer whereby participants used words like critique, judge and check to describe the 
purpose of their tool. Another example can found in the understanding layer where tools are normally 
employed to network, contribute and chat, while participants used words like compare, interpret and explain 
to describe the purpose of these tools. This goes out to show that at some level the different digital tools 
under investigation fall under broad categories that can be traced back to the six layers of a person’s thinking 
skills and that each tool addresses that specific skill at some level. The analyses layer was also noticed to 
have been characterized by words like compare, outline and find that were used by participants to describe 
the purpose of tools that require skills to review and question. The application layer that typically involves 
tools that require skills to reply, post and blog were described by participant with words like implement, use 
and execute. Finally, tools employed to text, message and tweet, associated with the lower order of a person’s 
thinking skills, the remembering layer, were described by the participants with words like name, find, and 
recognize. 

Table 1. Web 2.0 Tools Employed During the Digital Pedagogies Empirical Study 

Facebook to reach out to students at a level medium; 

Twitter to tweet educational related material; 

WhatsApp to communicate with other right away; 

Google hangouts to set group meetings; 

Skype to communicate in real time; 

Viber to communicate synchronously; 

Instagram to bookmark interesting stuff to share; 

Second Life to simulate virtual environments that can serve educational purposes; 

Diigo to save, organise & annotate online sites that are considered interesting, useful & important; 

Pinterest to share interesting photos; 

Weebly blog to post your thoughts and opinions for others to follow; 

Wikispaces wiki to store information and educational resources; 

Scoop to be able to reuse content from other interesting sites; 

Flashmeeting to be able to set realtime meetings with numerous people concurrently 

Merlot to access ready-made educational resources; 

Prezi to author some dynamic presentations; 

Slideshare to be able to share all your presentations; 

Quia to be able to set online polls; 

SurveyMonkey to develop online surveys; 

Moodle to host your lessons; 

Piratepad to collaborate online writing pad; 

Padlet as a common whiteboard over the web; 

Mindmeister interface for a group to collaborate creatively using mind maps; 

Vocaroo to voice recording; 

Screencast-o-matic to record a screencast; 

Edmodo environment to get students connected within a digital classroom; 

Imgur to save, use, reuse and share photos; 

Powtoon to create animated videos and presentations; 

Hipchat to hold meetings that include group chat, video chat & screen sharing; 

Coursera MOOC to register for a MOOC; 

Youtube to upload all your videos you want to share with students; _ 

The results for the second node were much more coherent as participants homed on the pros and cons of 
such tools irrespective of which specific tools they investigated. What emerged was the fact that, similar to 
any teaching aid, is these tools are employed effectively they lead to higher levels of communication and 
collaboration amongst the learners. On the other hand the participants pointed out that such tools, beneficial 
as they might be, do not serve all types of purposes and that an educator needs to skillfully judge precisely 
which tool to employ for a specific pedagogy. Creativity was one of the frequently used descriptors when 
participants highlighted the benefits, while frustration and steep learning curve were predominantly used to 
highlight the challenges that such digital tools can create. Another term that was frequently used to describe 
the benefits of such digital tools was the freedom and flexibility that they offer, however this same term was 
used by some other participants to describe a disadvantage. They argued that some learners are threatened by 
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such openness and lack of direction or guidance. What emerges from the analyses of this node is the fact that 
caution needs to be applied when employing such tools to ensure that the learners are at the right cognitive 
level of the tool itself in a way that they can easily master it and fruitfully benefit from it without any major 
effort. 

The third node produced much more open-ended results as participants proposed a plethora of 
recommendations, however it was clear that once participants were reluctant to make use of a tool that 
performed a similar functionality to a tool they had already mastered. Their recommendation highlighted the 
fact that an educator must employ a tool that s/he has already mastered her/himself, and rather than making 
use of a new unknown tool for the sake of novelty the educator should stick to what s/he is knowledgeable 
about. Additionally, given the number of tools available, a number of participants recommended that only a 
limited number of tools are employed that specifically perform a unique functionality not to confuse learners. 
Learners should be incrementally introduced to a tool allowing them ample time to let it sink in as well as 
enough time to venture, investigate and try the tool out in a safe environment. This recommendation given by 
a majority of the participants reflects their own experience with the tool as they struggled, enjoyed, and 
mastered the tool itself. Finally, the participants pointed out to ensure that the tool is strictly employed in line 
with a specific learning objective in mind rather than just simple used for the sake of novelty or originality. 
Such a recommendation brings us back to the original purpose of digital pedagogies and the educational 
rationale to employ tools or aids to facilitate the learning process and ameliorate the medium employed. 


5. CONCLUSION 

This empirical study as part of this research has shown that digital pedagogies are a complex and 
ever-changing entity that are necessarily part of the continuous professional development of all educators as 
new tools and techniques address the entire teaching spectrum. The potential of digital pedagogies has been 
highlighted by the feedback given by the participants while a number of point of caution have also been 
pointed out to ensure that such tools are appropriately employed with a clear educational purpose and in a 
safe pedagogical environment. Educators are required to continuously stay abreast of emerging tools and 
technologies ahead of their own students if possible cognizant that digital pedagogies in isolation do not 
guarantee an effective learning process, but that the educator’s classical role of guiding, facilitating and 
leading is and will remain imperative. 
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ABSTRACT 

In order to develop elementary school children’s environmental citizenship, the 21st century teachers have to improve 
their own environmental literacy, and electronic sensors can have an important role in that improvement. The research 
presented in this paper describes an environmental education project that uses electronic sensors to support future 
teachers in thinking globally, while sensing and acting in their local environment. Future teachers used sound sensors, 
temperature sensors, as well as carbon dioxide sensors, together with smartphones, to explore natural and urban systems 
in order not only to characterize these systems, but also to: i) understand ecosystems’ services and interconnectedness; ii) 
identify environmental problems; and plan related solutions in what concerns climatic comfort and air quality. The data 
acquired and produced in ecosystems’ explorations were selected and are presented in this paper, as well as the related 
interpretations. The learning results are related to: i) the local characterization of a natural ecosystem and of the teachers’ 
education school campus; ii) the differences and relations between Lisbon urban systems and the neighboring natural 
ecosystems; iii) the identification of interventions to improve environmental quality of the campus. This research 
contributed to validate the use of electronic sensors and smartphones as a useful strategy to produce environmental 
information about the explored ecosystems, to link local and global approaches in environmental education, as well as to 
support students’ actions, planned to enhance the Campus environment. 


KEYWORDS 

Sensors, Environmental Education, Teachers’ Education 


1. INTRODUCTION 

In the context of the development of fundamental competences for the 21 st century and of elementary school 
children’s environmental citizenship, this research emphasizes environmental citizenship and the use of 
ubiquitous digital tools, such as electronic sensors, aiming at supporting future teachers in sensing locally 
natural and urban ecosystems, to understand local and global ecosystems. 

New technologies together with transnational environmental problems are fundamental drivers of the 
definition of the skills and knowledge needed to the 21 st century (Scott, 2015). Furthermore, UNESCO 
(2017) stresses that the quality of education, including the content of the education provided, and the 
excellence of teachers, is crucial for achieving sustainable development, and all of the Millennium 
Development Goals (MDG). In this context, the integration of sustainability into education will support 
natural literacy and a consequent desire to protect and restore nature (World Business Council for Sustainable 
Development, 2010). 

The research presented in this paper is grounded on the idea that twenty-first century students must 
develop the ability to make sense out of significant and complex global issues, through an education centered 
on real world issues, and on problem, inquiry and project-based learning strategies, enhanced with new 
technologies (Scott, 2015). Accordingly, the research presented in this paper is part of the Glocal Act project 
(Knowing the global environment to act locally: From learning in natural spaces to urban intervention). 
Glocal Act is an education for conservation project, which aims to develop: i) future teachers learning in 
natural surroundings of Lisbon; ii) future teachers understanding of the ecosystems services and of the 
relations of these spaces with the city; iii) a consequent environmental intervention located in an academic 
Campus in this city. 
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In a subsequent stage, the Glocal Act project will support participant future teachers in adapting and 
implementing similar environmental activities with children, also aiming at environmental exploration and 
critical reflection to improve local environments. 

Following this introduction, the previous and related work will be presented, as well as the methodology, 
the late breaking results, and the conclusions. The last section includes the bibliographic references. 


2. SENSING LOCALLY WITH ELECTRONIC SENSORS 

In this paper, the authors present the late breaking results of the Glocal Act project, specifically the 
acquisition and interpretation, by students of a Teachers Education School, of environmental information of 
two ecological systems: a Lagoon near Lisbon and the outdoor space of the Lisbon Polytechnic Institute 
Campus. Electronic sensors are used in this research to support students of a teachers’ education school in 
thinking globally, while sensing and acting in local environments. 

2.1 Previous and Related Work 

The Globe project is a pioneering participatory project that started in 1995. In this Project, the data collected 
with sensors by schools are processed and presented for local and global interpretations (Globe Program, 
2017). Since the beginning of the 21st century, tools such as mobile devices (e.g. PDA and smartphones) and 
sensors (e.g. GPS, cameras, and sound and temperature sensors) have been used in participatory 
environmental activities. Ambient Wood (Rogers et al ., 2005), MobGeoSens in Schools (Kanjo el al, 2008) 
are examples of projects that built and used mobile devices to monitor real-world environmental parameters, 
in georeferenced environmental sense making activities. 

Environmental sensor kits have been multiplied, (McGrath, and Scanaill, 2014) with the increase of the 
daily use of smartphones with Internet access (Lerreira, Ponte, Silva and Azevedo, 2015) and the 
development of widely accessible sensors. As a result of the persistent use of digital and mobile technologies, 
most students today are natural researchers, since those technologies are an effective way to support 
independent and enquiry-based learning (Scott, 2015). This use of mobile technologies also allow for instant 
and reflective assessment (Scott, 2015). The two following projects are examples of such affordances. The 
Kids Making Sense program (Kids Making Sense, 2017) empowers youth to collect credible air quality data 
around their neighborhoods. Students can even use their data to identify local sources of air pollution and 
take actions to be part of the solution. This program was developed by air quality scientists and educators. In 
the USense2Learn project, children used environmental sensors and smartphones to create georeferenced 
multisensory information. Children assessed the environmental quality of their schoolyards and shared the 
developed knowledge into the classroom, and to other classrooms, using Google Earth (Silva, Lopes, Silva 
and Marcelino, 2010). 

In this research, students of a teachers’ education school used mobile sensors together with mobile phones 
and Google Maps to sense a natural system of the neighborhoods, and the outdoor environment of their 
Campus, to reflect critically and globally on the acquired data, aiming to improve the Campus environmental 
quality. 

2.2 Methodology 

This research is being developed in the context of initial teachers’ education in a Higher Education School in 
Lisbon, Portugal. It uses a qualitative methodology, centered on a case-study approach, in which the teacher 
was also a researcher, making it possible participant observation. The case-study activities are organized in 
three phases: 

1. Use, by a class of 15 students (14 female and 1 male), of the carbon dioxide sensor and of the 
temperature sensor, together with smartphones/tablets, to acquire environmental data, while exploring a 
Lagoon natural system, close to Lisbon (31km) - Lebruary 2017; 
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2. Use, by a class of 16 students (2 male, 14 female), of the sound, carbon dioxide and temperature 
sensors, and of the digital microscope to explore the outdoor environment of their academic Campus. The 
results of the explorations were mapped by the students in a collaborative map of Google Maps - September 
2017; 

3. Critical interpretation of the environmental information acquired in the two systems, by two classes 
(one of 16 students, 2 male, 14 female, and another one with 23 students, 21 female, 2 male), and planning of 
interventions in the Campus- September 2017. 

Future teachers worked in groups of four to five students, and used a PASPORT Weather Anemometer 
Sensor (PS-2174), a PASPORT Carbon Dioxide Gas Sensor (PS-2110), as well as a portable USB Digital 
Microscope. 

2.3 Late Breaking Results 

The data, acquired by students with the sensors in the Lagoon natural system, were presented in graphics 
(figure 1), while the sensorial and quantitative data acquired in the Campus system were georeferenced and 
presented, using images, numbers and text (figure 2 and table 1). In figure 1, the graphics show that the 
temperature and C02 values are changing in time, since the groups with the sensors were moving, and 
carbon dioxide values are low most of the time, with peaks when the group went into the birds’ observatories. 
In the two placemarks opened in Figure 2, the groups of students registered the sound level (mean 69,43dBC, 
maximum 74,44dBC) and the carbon dioxide data (mean 586ppm, maximum 1164ppm), acquired near the 
main road. This road was identified as a source of noise and air pollution, since those values far exceed 400 
ppm, the mean value of COt in atmosphere (WMO, 2016), as well as the limit of 60dBC for the ancillary 
learning space (Lilly and Wowk, 2010). These values also exceeds the ones further from the road. The 
canteen is an interior noisy place. 


Air temperature 

C0 2 concentration in air (ppm) 

(°C) 


3000 

40 


20 

1000 (I ft 1 

0 

otooLOOLOOLOOLnoLOOLOOLnOLn 

0 

67 

134 

201 

268 

335 

402 

469 

536 

tflfnCliDNClinrMMihHCO'tHb^O 

HHNrOffl'tihifliDNNCOOlOlOH 

rH r-1 


Figure 1. Data of Air Temperature and Carbon Dioxide in Air Vs Time (S), Acquired in the Lagoon Natural System 



Figure 2. Collaborative Map of the Outdoor Environment of the Camp, with the Several Placemarks Created by Students. 
Two of the Placemarks, Near the Main Road Are Showing Their Content 
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Table 1. Selected Data Acquired by Students and Mapped Using Placemarks 


Placemar 

k 

Carbon dioxide (ppm) 

Placemark 

Sound level (dBC) 

Mean 

Maximum 

Mean 

Maximum 

1 

371 

425 

4 

69,43 

74,44 

2 

336 

493 

5 

50,10 

70,97 

3 

586 

1164 

6 

(canteen) 

68,37 

97,16 


Based on the acquired data, students were able to: i) use the sensors without difficulties, in a engaged 
way; ii) recognize the low levels of CCL (below the planetary mean) of the Lagoon system, while relating 
these low levels to the photosynthesis and the CCL peaks to the lack of quality of the air in the human 
crowded observatories of birds; iii) identify environmental problems of the Campus, such as thermal 
discomfort, sound pollution, and CCL pollution, while recognizing their causes; iv) envision a set of solutions 
to the identified problems, namely to plant trees to create shadow (they used the sensors to prove the lower 
temperatures at the shadowed places), and to place a barrier, “for instance of cork”, to protect the Campus 
from the traffic noise. 

Teacher mediation was needed to: i) inform the global average concentration of CO 2 in Earth's 
atmosphere, to allow comparisons and deduce explanations; ii) provide information on the consequences of 
sound levels to human beings, also to allow comparisons and deduce explanations; iii) call attention to the 
similar thermal amplitude verified when comparing temperatures at shadow or at direct sunlight both in the 
Lagoon system and in the Campus; iv) support students’ reflections on the potential of the Lagoon system to 
improve Lisbon’s environmental quality, specifically in what concerns air pollution; v) inform about the 
possible use of vegetation, bushes and trees, as acoustic barriers; vi) supply information about the indigenous 
species, and the invasive species, to facilitate the creation of more complete solutions by students. 


3. FINAL REMARKS 

The late breaking results of this work contributed to the validation of the use by future teachers of electronic 
sensors, together with smartphones, as a useful strategy to produce environmental information about a natural 
and an urban ecosystem, and to critically reflect on it. Teacher mediation was important to support students 
in using such information in linking local and global approaches to environmental problems, and in planning 
actions to enhance the Campus environment. Future activities will include a more extended (in space and 
time) characterization of the Campus and of other natural systems near Lisbon, by future teachers and 
schoolchildren, to improve local sensing, global thinking and local interventions. 
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ABSTRACT 

Digital portfolios can provide a space where evidence of a learner’s competency is stored and digital badges can be used 
in their portfolios as valid indicators of accomplishment, skill, knowledge, or interest. The authors issued ‘digital badges’ 
to our students who successfully completed the modules of a Medical English Terminology course we developed on 
Moodle by using its badge function. The badges which students earned for the course were designed to function as a 
validated indicator of students’ achievements to demonstrate to their teachers. We anticipated that the use of badges 
would not only help the learners confirm their achievements for the course but also help motivate them toward further 
autonomous study. Through a reflective questionnaire about the use of badges, about 77.5% of the students found the 
badge assessment system comprehensible and the great majority of students (87.2%) were satisfied with their study 
through this course. The questionnaire results also showed that 67.6% of the students found the use of badges helpful in 
confirming their course achievements and 63.7% of them found that earning badges helped motivate them toward further 
autonomous study. These findings indicate that while there is still room for improvement, the use of digital badges has 
the potential to provide students with opportunities to celebrate their achievements and enhance their learner autonomy 
for online self-study courses. 


KEYWORDS 

Digital Badges, Moodle, Indicator, Motivator, Autonomous Study 


1. INTRODUCTION 

In monitoring students’ learning outcomes, teachers have long been using a range of traditional measures 
such as allocated grades or credits. However, these specific measures are institutionally controlled, awarded 
according to achievement and do not always represent a learner’s true effort or illustrate their progress. In 
e-learning environments, learners have more choice in the time and place the learning will occur. While 
teachers design digital content and activities to achieve identified objectives, the ultimate responsibility of 
achieving those outcomes is transferred from the instructor to the learner. In these more personalized 
environments, learners need to be more self-motivated and self-directed (Clayton, 2009). 

Digital portfolios can provide a space where evidence of learner’s competencies and achievements are 
stored and systematically evaluated (Fiedler, et al, 2009). Digital badges have been designed to indicate 
progress at the point of achievement and are used as valid indicators of accomplishment, skill, knowledge, or 
interest. 

At Shimane University, the authors implemented the idea of issuing ‘badges’ in a Medical English 
terminology course we developed using the badge function of Moodle, a popular course management system. 
The course included 13 sections, each of which had three types of terminology quizzes. Students who 
successfully completed all the quizzes of each section were issued a ‘section badge’ and when students had 
earned all of the section badges and passed the final test, they were issued a “course badge.” The badges were 
displayed in a learner portfolio, which functions as a validated indicator of progression. We anticipated that 
the use of badges would not only help the learners confirm their achievements, but also help motivate them 
toward further autonomous study. 


171 



ISBN: 978-989-8533-71-5 © 2017 


2. AWARDING BADGES 

Badges actually have been used in many ways to reward achievements. For example, boys and girls in 
scouting programs earn badges when they have progressed their skills or attended special events. They are 
given a badge to wear on their uniform to show others what they have achieved. These badges are a source of 
pride to the scout members who earned them and a way to demonstrate what they have accomplished. In a 
very similar way, video games also award badges to provide players with rewards for completing various 
levels. 

This idea has been applied to e-learning and there exists a variety of e-learning courses which award 
badges to users when they have achieved targeted levels or skills (Poldoja, et al, 2016). The use of badges in 
educational settings has substantial potential for future growth. 

In this study, the authors hypothesized that we could enhance learner autonomy by implementing digital 
badges for the online courses we developed. We expected that using badges would help learners confirm 
their achievements, help them engage more with the courses available and participate more autonomously in 
learning activities. 

We implemented the idea of issuing ‘badges’ in a Basic Medical English Terminology course we 
developed using the badge function of Moodle. The course was designed to help lst-year medical students of 
Shimane University review 1,000 basic, frequently-used terms which are keys to better understanding 
up-to-date information about health-related issues, medical conditions, diseases, symptoms and treatments. 
The course included 13 sections such as body parts, symptoms, medical devices, facilities and abbreviations 
(Figure 1). Each section included three types of quizzes such as a multiple-choice quiz and a matching quiz 
as shown in Figure 2 and 3. 




Figure 1. Course Page Figure 2. Multiple-choice Figure 3. Matching Quiz 

Students first chose a section in the course and then worked on the quizzes they chose in that section. To 
pass each quiz, they had to meet certain criteria we set. When students successfully passed all the quizzes of 
each section, they were issued a ‘section badge’ (Figure 4). When students had earned all 13 section badges 
and had passed the final test, they were issued a ‘course badge’ (Figure 5). The badges were displayed in a 
learner portfolio (Figure 6), which functioned as a validated indicator of the learner’s progress. The badges 
earned in the portfolio demonstrate the skills and knowledge the learners have acquired. We anticipated that 
the use of badges would not only help the learners confirm their achievements through the course study, but 
also help motivate them toward further autonomous study. 
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Figure 4. Section Badge 



Figure 5. Course Badge 


Figure 6. Learner Portfolio (List of Badges) 


3. EVALUATION 

In February, 2017, after all the students had completed the medical terminology course, their feedback 
regarding the badges was collected from 102 students (RR=100%) through a questionnaire, which included 
the following questions: 

(1) To what extent was the badge-based assessment system comprehensible? 

(2) To what extent are you satisfied with your medical English study though this course? 

(3) To what extent was the badge system helpful in checking your achievements? 

(4) To what extent did the badges influence your learning motivation? 

The results of the questionnaire showed that about 77.5% of the students found the badge assessment 
system comprehensible (Table 1). They also showed that the majority of students (87.2%) were satisfied with 
their study through this course (Table 2). 

Table 1. Comprehensibility (n=102) Table 2. Learner satisfaction (n=102) 



n 

% 


n 

% 

Very comprehensible 

36 

35.3% 

Very satisfied 

27 

26.4% 

Comprehensible 

43 

42.2% 

Satisfied 

62 

60.8% 

Neither 

20 

19.6% 

Neither 

10 

9.8% 

Not so comprehensible 

1 

1.0% 

Not so satisfied 

1 

1.0% 

Not comprehensible at all 

2 

2.0% 

Not satisfied at all 

2 

2.0% 


As for the expected function of badges as an indicator of student achievement, 67.6% of students found 
the badges were helpful but about a third of them were unsure about the effects (Table 3). As for learner 
motivation, 63.7% of students found that earning badges helped motivate them toward further autonomous 
study, while the rest found it didn’t help much with their autonomous study (Table 4). 

Table 3. Badges for Checking Achievements (n=102) Table 4. Badges for Learner Motivation (n=102) 



n 

% 


n 

% 

Very helpful 

20 

19.6% 

Very motivating 

19 

18.6% 

Helpful 

49 

48.0% 

motivating 

46 

45.1% 

Neither 

21 

20.6% 

Neither 

21 

20.6% 

Not so helpful 

8 

7.8% 

Not so motivating 

8 

7.8<>/o 

Not helpful at all 

4 

3.9% 

Not motivating at all 

8 

7.8% 


Overall, our students had positive impressions of the badges in terms of comprehensibility and learner 
satisfaction, however, the results regarding the awarding of a badge as an indicator and a motivator were not 
as good as expected. 
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In addition to asking the questions mentioned above, we asked our students to give comments about the 
use of badges and the following is a selection of some of the comments that were given: 

> “The course was very demanding but I found it very helpful with my review of the medical 
terminology.” 

> “I think this digital learning environment will give us a more holistic view of our progress.” 

r “The badges I earned should be more clearly displayed on the course.” 

> “The design of the badges, especially the course badge should be improved. It would be more 
motivating if it was a trophy.” 

While students mentioned the benefits of earning badges and the appeal of the digital learning 
environment, some feedback showed that problems still remain such as there is room for improvement in 
terms of the display of the badges as well as in the badge design. 


4. CONCLUSION 

In this study, the authors developed a Basic Medical English Terminology course on Moodle using the badge 
function and investigated if the use of the badges would help learners confirm their achievements through 
course study and help them participate more autonomously in learning activities. 

The initial findings from the survey on students’ perceptions of their study through the course indicated 
that as a whole, the majority of the students (77.5%) found the badge-based assessment was comprehensible 
and the great majority of them (87.2%) were satisfied with their medical terminology study. The survey 
results also revealed that two-thirds of the students (67.6%) found the badges helpful in checking their 
achievements while about one-third of them (32.4%) were still unsure about the effects of the badges. With 
regard to the function of the digital badges as a motivator, the same basic results were found. About 
two-thirds of the students (63.7%) found that earning badges motivated them with their study. The comments 
given by the students about the use of badges showed that there is room for improvement in terms of the 
display as well as the design of the badges. 

We are conscious that we need to further investigate how awarding badges actually helps students check 
their progress and how it helps them become motivated to study autonomously. We also need to modify the 
design of badges and the layout of the portfolio. 

Digital badges are a response to an emerging educational landscape in an increasingly digital world. They 
provide a framework that facilitates learning in new contexts and provides opportunities for learners to mark 
and celebrate their achievements. The challenge for the authors is to rigorously review the use of ‘digital 
badges’ to ensure they meet the learner autonomy requirements. We will continue to use digital badges in our 
online language courses to further enhance self-directed learning and learner autonomy among our students. 
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ABSTRACT 

The goal of this study was to explore the educational potential of Minecraft. This project was conducted with 118 
elementary-school students from Canada during the 2016-2017 school year. To explore the educational potential of 
Minecraft on the students, we designed a “Minecraft challenge” for students. 10 game levels of ascending difficulty were 
created for them, each containing 3 sublevels to pass before moving up to the next level. In order to advance to the 
“Minecraft Master level”, the students had to complete a total of 30 tasks. A moderator specialized in Minecraft 
gameplay was present during the gaming sessions, which took place at the schools. Ten different types of data collection 
tools, such as online surveys, interviews, classroom observations, “think aloud” interviews during Minecraft gaming 
sessions, student-generated Minecraft “creations”, etc. hi this study, we first report on how Minecraft was used with those 
118 elementary-school students. We then highlight the educational potential of Minecraft by describing how the use of 
Minecraft benefitted students in various aspects of their learning. These positive outcomes include, but are not limited to: 
an increase in motivation, the development of collaboration skills, the learning of computer programming, and the 
development of other computer science competencies. 


KEYWORDS 

Minecraft, Educational Games, Motivation, Collaboration, Creativity 


1. INTRODUCTION 

A study called “Can Minecraft Transform Schools?” might ring some alarm bells for parents. And their 
concerns would be justified, given the immense appeal of this game for youngsters. It is the second most 
popular video game of all time, with over 100,000,000 copies sold. The chief concern for parents is that these 
games will have negative impacts on their children: that it will generate more conflicts at home and more 
violence in general. What parent has never asked their child to quit playing a video game, or has never felt 
queasy about inappropriate content? So, parents and teachers are right to be worried, for the most part. 
Furthermore, their disquiet has been fed by the media and studies that have spread fears of how technology 
abuse can harm child development. That said, aside from the widely held view that technology abuse can 
have negative consequences, people are largely unaware of the enormous educational potential of Minecraft. 
In fact, this video game is rapidly catching on at schools, and an educational children’s version (Minecraft 
Education Edition) was released in the fall of 2016. Since 2013, American 1 and Swedish 2 * schools have been 
systematically integrating Minecraft into their schools, and it is being used around the world to teach 
science, urban planning, and foreign languages. Closer to home, in Montreal, a number of schools have 
joined Mission 375, 4 a contest in which students use Minecraft to reproduce historic sites and events around 


1 https://www.washingtonpost.com/lifestyle/kidspost/minecraft-spawns-classroom-Iessons/2013/03/14/717aed66-87b8-l Ie2-98a3- 
b3db6b9ac586_story.html 

2 http://www.pcgamer.com/minecraft-becomes-a-compulsory-class-for-swedish-school/ 

1 https://minecraft.net/fr/ 

4 http://plus.lapresse.ca/screens/d787353b-43e3-4342-b537-8c379d9de0fd%7C_0.html 
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Montreal in celebration of the city’s 375th anniversary. As a bonus, masses of educational applications and 
experiences using Minecraft are available on online sites and forums. Accordingly, and in line with our 
research program, we felt it important to shed scientific light on the educational potential of Minecraft in 
order to provide a deeper understanding of the impacts on young learners. 


2. IS THERE A PLACE FOR VIDEO GAMES AT SCHOOL? WHAT THE 
RESEARCH SAYS 

For many decades, researchers have underscored the value of games for educational purposes 
(Dewey & Deledalle, 1983; Piaget, 1959; Winnicott, 1975). Today, with the advent of digital technology, 
video games are earning profits of over 100 million dollars annually, and are now considered a major 
“cultural industry.” The good news is that studies have repeatedly shown that video games can help students 
learn (e.g., Baranowski et al., 2003), with positive effects on the cognitive, affective, and psychomotor 
domains (Shaftel, Pass & Schnabel, 2005). A comprehensive literature review on the educational impacts of 
video games showed that they can increase learning in youth by at least 12%, with improved hand-eye 
coordination, problem-solving skills, and memory, among others (see Clark, Tanner-Smith & Killingsworth, 
2013). Some studies have demonstrated that surgeons who regularly play video games are more effective at 
their job (Rosser et al., 2007). In short, the evidence indicates that video games contribute directly to 
academic success. Nevertheless, the reality is a little more complicated, and games, particularly video games, 
have not yet achieved hallowed status in the education world. There is no doubt that education systems are 
undergoing a digital transformation in Quebec, across Canada, and abroad. However, some questions have 
been raised, notably concerning the discrepancy between the presence of technology in daily life and in the 
classroom. Besides having a bad reputation, video games suffer from this discrepancy between society and 
school. Digital resources are now an assumed feature of school life, and there is little doubt that some video 
games and apps for tablets and cell phones have educational value. For example, apps such as Scratch and 
ScratchJr allow students to have fun creating virtual stories and video games as they learn how to code. Since 
2016, other newly introduced apps such as Swift Playgrounds definitely fall into the educational category. A 
recent development is the holding of commercialized tournaments using video games like Assassin’s Creed, 
which teaches students history (Joly-Lavoie & Yelle, 2016) and geography (Gilbert, 2017). To gain a better 
understanding of the educational potential of digital resources (i.e., apps), two characteristics could be 
assessed. The first is the educational potential (what do learners derive from playing the game?), and the 
second is the fun potential (how much do players enjoy playing it?). In other words, a game that scores 10 out 
of 10 on educational potential (it elicits a lot of learning) but scores only 1 out of 10 on tun potential (it’s not 
very interesting) would probably be less valuable as an educational resource than a game that scores 5 on 
educational potential and 9 on fun potential. It wouldn’t take much persuading to get students to play a really 
fun game. We therefore set ourselves the rather pleasant task of classifying a number of games, educational 
resources, and apps on a two-axis graph. The results are thought-provoking. For example, we attributed a fun 
score of 9 out of 10 to Minecraft (the second most popular video game ever) as well as 7 out of 10 for 
educational potential. In contrast. Grant Theft Auto (GTA) was rated only 1 for educational potential (a few 
minutes of play sufficed us to realize that the skills acquired in this game can’t be transferred to actual 
driving), but received a 9 for fun potential, and indeed, it was one of the most popular games in 2017. 

According to our classification, Minecraft rates the highest for combined educational and fun potential. 
The Minecraft Education Edition appears to be particularly suitable for helping students develop their 
teamwork skills, computer skills, and coding skills. In addition, it is highly entertaining. We then decided to 
push our hypothesis further as part of an exploratory study conducted to gain a deeper understanding of the 
educational potential of Minecraft. 


5 https://scratch.mit.edu/ 

6 https://www.apple.com/ca/fr/swift/playgrounds/ 
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3. METHOD 

We examined the use of Minecraft Education Edition in two schools in the Greater Montreal Area (Quebec, 
Canada). The participants were 118 elementary school students in grades 3 to 6. To investigate the effects of 
Minecraft gaming on the students, we adopted an action research approach in which we applied several 
newly created game levels. Specifically, we designed 10 game levels of ascending difficulty, each containing 
3 sublevels to pass before moving up to the next level. In order to advance to the Minecraft Master level, the 
students had to complete a total of 30 tasks (Figure l). 7 A moderator specialized in Minecraft gameplay was 
present during the gaming sessions, which took place at the schools. 



Become the Minecraft Master 


Leuels 


01 Yellow 


♦♦ 

♦♦♦ 


Tasks 


Create a new world 

Set the gamepby commands 


Leuels 



Tasks 

Bi4d a wooden pickaxe 
Biddanoven 

BuM a torch (Start by makng coat and 
then create the torch) 


1 


♦ Move throughout the wortd 

1 


02 

Orange 

♦♦ Go into and get out of water 

♦ ♦♦ Breefcaoie 

07 

Bronze 




Figure 1. The Ten Advancement Levels in Minecraft Education Edition 


During the experiment, we gathered data using the following 10 methods: 

• Survey questionnaires (n = 4) administered to all participants (n = 128) 

• Individual, semi-directed interviews (n = 6 X 30 minutes) outside of supervised game time 

• Short individual interviews (n = 118X5 minutes) during supervised game time 

• Group interviews with students during Minecraft gaming sessions (n = 3) 

• Videotaped observations (n = 6 X 75 minutes) during supervised game time 

• Videotaped observations of think aloud protocols during Minecraft gaming sessions (n = 3 X 
30 minutes) 

• Individual interviews with teachers and moderators during supervised gaming sessions (n = 6) 

• Tracking of students’ advancement through the game levels 

• Weekly journals kept by the Minecraft moderators (n = 14) 

• “Digital footprints” (Jaillet & Larose, 2009), i.e., student-generated Minecraft products. 


7 The levels are available at: http://www.karsenti.ca/code/minecraft-education-edition/ 
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4. RESULTS 

The results of this exploratory study highlighted a number of educational benefits of using Minecraft in class. 
The main benefits are listed as follows: 

1. Increased overall motivation toward school 

2. Stronger computer skills 

3. Greater creativity 

4. Increased feelings of academic self-efficacy 

5. Positive learning environment 

6. Improved reading skills 

7. Improved writing skills 

8. Autonomy development 

9. Increased collaboration between students (many chose to work in groups) 

10. More mutual assistance between students (students tended to help each other troubleshoot gaming 
issues) 

11. Improved computer programing and computational logic skills (Minecraft’s advanced levels elicited 
basic programming skills) 

12. Improved problem-solving skills 

13. Improved informational research competencies (to accomplish certain tasks) 

14. Development of math concepts (perimeter, volume, calculation of required resources and numbers 
of blocks, etc.) 

15. Deeper understanding of scientific concepts (e.g., identifying the elements required to start a fire, 
grasping basic agricultural principles in order to accomplish certain tasks) 

16. Greater perseverance in the face of adversity (Minecraft tasks could be highly challenging, and 
some tasks had to be repeated to improve productions) 

17. Better understanding of history (especially when re-creating scenes from past decades or centuries) 

18. Improved ability to follow instructions (methodological skills) 

19. Greater academic self-esteem 

20. Improved oral communication skills 

21. Improved ability to create high-quality products 

22. Improved social skills 

23. Improved information and communication technology (ICT) skills 

24. Improved information organization skills 

25. Improved inductive and deductive reasoning. 

Along with all these benefits came a few challenges. For example, technical glitches interrupted the fun 
far too often. Some workstations disconnected in the middle of games, which the students naturally found 
unnerving. Updated equipment in good repair and with adequate capacity would go a long way to avoid this 
nuisance problem. Nevertheless, even though Minecraft could be rather challenging for some students who 
were new to the video game, all the participants made rapid progress in their gaming skills. 


5. CONCLUSION 

The results of this exploratory study of 118 students in two Quebec elementary schools confirm that 
Minecraft has real educational value. Notably, gaming allowed the students to fully engage in activities that 
were both educational and fun, and the outcomes were many and positive. The advancement levels (see 
Figure 3, Become the Minecraft Master) encouraged the students to progress step-by-step through a series of 
skills. Similar initiatives implemented in other schools in Quebec, Canada, and abroad would undoubtedly 
produce similar positive outcomes, as demonstrated in Sweden, a pioneer in the educational use of Minecraft. 
Considering the 25 benefits for students that we identified in the present study, there is every reason to 
believe that Minecraft projects could be equally beneficial for students elsewhere. Finally, along with the 
benefits for students, certain limitations and guidelines need to be taken into account. Unless they are 
supervised, children may not have the self-discipline to stop playing by themselves. Knowing this, we 
implemented a structured Minecraft program in a school setting with a moderator present during gameplay. 
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As a general rule, a good balance should be struck between video gaming and other types of activities. 
Because obsessive gaming may take precedence over constructive learning, it is up to parents and teachers to 
weigh the merits of the different ways that children spend their time. Therefore, in order to derive the full 
educational potential of Minecraft, it must be used judiciously. 
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ABSTRACT 

When students are engaged in the process of creating a digital story, they synthesize a variety of literacy skills for the 
authentic product: researching, writing, organizing, presenting, interviewing, problem-solving, assessing, as well as 
employing interpersonal and technology skills (Baggett, 2007). With all these skills to be developed among students, 
digital story creation is found a highly valuable undertaking. This paper lays down the perceptions of college students 
about digital technology. Specifically, this research examines the rigors and benefits of digital story creation pertaining to 
1) its influence towards academic performance, and 2) its role in promoting communicative competence. Hence, this 
paper investigated students’ experiences in creating and using digital story books as used among classrooms in the 
tertiary level. Surveys, interviews, reflective journals and analysis of created digital stories served as data gathering tools 
in completing the paper. 


KEYWORDS 
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1. INTRODUCTION 

Sadik (2008) considers digital story telling as a meaningftil technology-integrated approach for engaged 
student learning. This poses a huge truism in that nowadays, it is not easy to capture the learners’ attention; 
more so that psychologists claim that typical attention span of lasts from ten to fifteen minutes only. Hence, 
something has to be done with the kind of learners today, learners who are so adept and equipped with 
technological know-how. Teachers then need to keep up with this fast changing trend. 

However, Jacobsenn (2001) believes that many teachers worldwide are not able to adopt technology for 
teaching and learning tasks, and the gap of technology presence in schools and its effective use is too wide. 
Many teachers believe that technology integration is a difficult, time consuming and resource-incentive 
endeavor and is therefore more trouble than it is worth (Heingold & Hadley 1990) as cited by Sadik (2008). 
These statements were proven true among some instructors of Capitol University. Most of the time, teachers 
use chalk-talk as a teaching strategy. However, majority of the teaching populace were into technology 
integration. Basic English and Literature classes made use of Digital Storytelling. 

Digital story telling is an innovative, technology-based method by which 21 st century students utilize 
technologically advanced resources to produce meaningful stories and presentations that in turn allow for an 
enriched co-construction of knowledge (La Lrance, 2012). La Lrance further cited Xu, Park and Basek (2011) 
who described three major elements of digital story telling: flexibility, universality, and interactivity with 
regard to community formation. These researchers further posited that flexibility is construed as a non-linear 
fashion for it allows the story teller a wide array of communicative options framed in a technologically based 
pedagogy. Universality on the other hand is seen as a result of the widespread dissemination of recording 
technology which has become vastly available today. Lastly, interactivity is distinguished as a convenient 
means of material and information exchange. 
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This paper then explored digital storytelling and creation among Education students of Capitol 
University. Teachers used the strategy to introduce topics relative to the course such as writing, multimodal 
composing and geographical setting. As observed, digital storytelling and creation aided tertiary level 
students to prepare them for leading roles in educational contexts. It somehow engaged these learners and 
stimulated the reflective learning in them. 


2. METHODOLOGY 

The subsequent headings describe the phases that this research underwent. 

Population Identification 

The study regarding digital storytelling as a learning tool was conducted first semester of school year 
2006-2007. The populace consisted of 68 students participating in an English Literature class as part of the 
course curriculum. Students were contacted after the course ended to request access to their reflections for 
this study. Fifty five chose to participate in the research. Among the participants, fifty were female and five 
were male. The respondents were between the ages of 16 and 20. 

Data Collection 

Archival data in the form of student reflections were collected as part of the course requirement. No right 
or wrong answer for purposes of the reflection was dinned to minimize non-participation of those who were 
not fond of writing. This was done to earn valuable feedback from the group. Hence, these were accessed by 
the researcher with assent from both teacher and student. A random interview was moreover conducted 
which served as basis for the analysis of the digital story creation. Demographic queries involving age, 
gender and field of specialization were asked during the survey-interview. Blizzard’s 2002 data collection 
model was dished up as a research pattern. 

The rationale of this paper explored the pedagogical benefits of digital storytelling. Therefore, it took 
course on avenues like student engagement, educational outcomes, and teacher-learner perception. 


3. FINDINGS AND CONCLUSIONS 

This paper investigated the influence of digital story creation among college students. It focused on exploring 
the latency of digital story telling as an instructional sUategy and how it influences student engagement and 
student outcomes such as describing academic performance. 

As mentioned by Sadik (2008), the use of technology is only effective if the teachers have the expertise to 
customize the use of technology for story creation. This was observed as a positive outcome of the paper. The 
teacher respondents were indeed well versed with technology. They had created much from the background 
knowledge of their learners. All praises were gathered from the fifty five respondents regarding the sttategy 
used (digital storytelling and in turn creation). 

The English Literature Teachers additionally started by giving an orientation among their students 
followed by workshops during the first two weeks of classes to support and engage them in the final project. 
Workshop one objectified the concept of digital storytelling which recalls past experiences with digital 
sound, video and storytelling. The moviemaker software was introduced in the second workshop. 
Interestingly, the learners found this topic splendidly enthralling. Their reflective journals showed thumbs up 
signs. During the workshop, moviemaker-digital-story-creation tiled up the topic. In the activity sessions, 
topics were chosen by students for the digital story creation which was evaluated by the teacher concerned 
for suitability. 

A huge impact of the strategy utilized in the entire semester was noted. Students performed exUemely 
well in the class activities making use of the digital storytelling and creation technique. They beamed with 
pride presenting their laid out stories for the week, more so when they manipulated the computer laboratories 
for the digital creation part. Indeed they rated high in the course as manifested in their instructors’ records. 
Several skills were enhanced such as writing, design, library and research, technology and communication. In 
addition, digital storytelling helped students with tasks they formerly found very difficult including spelling. 
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sentence formation and building, and forming the whole body of a text. Hence, this integration of technology 
in digital creation assisted students to overcome their writing problems. 

Furthermore, teachers observed that students were learning without realizing. Provided that students are 
clearly informed about the task that is required of them, digital storytelling is useful as an all-round skill 
development tool; the use of digital storytelling can therefore reinforce various complementary skills. As a 
result, teachers had positive attitude towards the use of DST as a teaching tool in their classrooms. Both 
teachers and students had the opportunity to improve their technological skills which includes the electronic 
devices necessitated. They auxiliary stated that Digital Story Telling may not only be used among English 
classes but would prove useful even in History and the Social Science subjects; Mathematics and the 
Humanities. 

Furthermore, teachers confirmed that the use of digital stories in education is beneficial for Universities 
receiving foreign entrants. The ability for expression through visual media rather than words facilitates 
communication for new students and builds their confidence. In addition, teachers fulfilled the role of 
facilitator, consultant and could scaffold the learning process more effectively when they used digital 
storytelling in class. 

Lastly, the new knowledge generated by this research can inform future educational policy. A number of 
story development models had been created in the past to help educators achieve better learning outcomes 
with the intent to promote communicative competence yet none provide a holistic pedagogical framework for 
reeking Communicative competence during the various stages of learning. This research presented an 
e-learning DST technique that lumps up communicative competence development among learners. Learners 
here were found to have fully enjoyed the said DST as they developed their communication skills to the 
maximum. 
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ABSTRACT 

Feedback on assessed work is invaluable to student learning, but there is a limit to the amount of feedback an instructor 
may provide. Peer feedback increases the volume of feedback possible, but potentially reduces the quality of the 
feedback. This research proposes a model of collaborative peer feedback designed to increase quality of peer feedback, 
and describes its implementation in an undergraduate mathematics module. The implementation includes the 
development of bespoke software to automate administrative tasks. 


KEYWORDS 

Peer Assessment, Group, Feedback 


1. INTRODUCTION: FEEDBACK & PEER FEEDBACK 

Educators seeking to improve their effectiveness are well advised to consider offering more feedback to 
students. Indeed Hattie (1987) concluded “Feedback is the most important single influence on student 
achievement” and Black & Wiliam (1998) opined “Feedback has extraordinary large and consistently 
positive effect on learning compared to other aspects of teaching.” Rice et. al. (1994) note that students are 
also aware of the value of feedback to their learning, “mathematics students want specific, detailed, 
facilitative feedback”. In this context, it seems clear that a great deal more high quality feedback would be 
beneficial for students. However, providing high quality feedback is a significant drain on instructor time, 
especially in large classes. 

Faced with restrictions on instructor time, a popular substitute for feedback provided by the instructor is 
feedback provided by other students, so-called “peer feedback”. 1 The advantages are many: not only is the 
task of giving feedback crowd sourced, but those giving the feedback also stand to benefit, Dochy et. al. 
(1999), Ambrose et. al. (2010). By critically engaging with the work of another student, the feedback 
provider is forced to check data, weigh arguments and appraise evaluations—different skills to finding data, 
constructing arguments, and evaluating methodologies, but tasks that mutually reinforce. Moreover, if a 
student has completed an assignment and is then asked to provide feedback to another student on that same 
assignment, they are forced to engage with the same material at multiple levels, thereby bolstering 
knowledge acquired and attaining deeper learning outcomes, Somervell (1993). 

There are also disadvantages to peer feedback compared to the feedback that an instructor can provide. 
The most pernicious is the quality of the feedback. It may not be that every student is equally motivated to 
provide feedback to the best of their ability, which necessarily disadvantages the students receiving weaker 
feedback. Even the most dedicated student will likely provide lower quality feedback than the instructor, 
were they unconstrained by time, would be able to provide. The student receiving a peer’s feedback is likely 
aware of this, and may even underestimate the quality of the peer feedback, so may engage with peer 
feedback less positively than with feedback provided by the instructor. 

Anonymity in assessment has many advantages, even when the assessor is the instructor, not least of 
which is reducing implicit bias. In peer feedback, reducing implicit bias is also helpful, but there is a still 
more important reason for anonymising peer feedback. If the feedback provider knows the identity of the 


1 Although peer assessment with stakes is a popular model, for example Forbes & Spence (1991), we will not discuss assessment with 
stakes in the present work. 
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student whose work they are assessing, it may cause some embarrassment for the assessed student, which 
makes students more likely to engage in avoidance behaviours, hampering their learning, Friedel et. al. 
(2002). If a student knows which other student gave them poor quality feedback, they may be less willing to 
work constructively with that student in future collaborative assignments. Therefore, as far as possible, peer 
feedback should be double blind. 

This research seeks to address some of the shortcomings of traditional peer feedback models that were 
described above. 


2. PEER FEEDBACK IN THE CONTEXT OF LEARNING OBJECTIVES 
OF THE PROOF MODULE 

Proof is a mathematics module pitched to second year students in a highly competitive undergraduate college 
of the liberal arts and sciences in Singapore. The module serves as a gateway to higher level modules in 
mathematics, computer science and statistics, providing a survey of foundational university level 
mathematics and introducing myriad advanced concepts and notation. But the main learning objectives of the 
module are centered on the writing and critical reading of proofs. Proofs are the means by which 
mathematicians convince one another of facts (or theorems), so it is as essential that a prospective 
mathematician, computer scientist or statistician be fluent in proof as it is for a budding historian to be able to 
write an essay or chemist a lab report. A challenge is that mathematics students have experienced only the 
palest facsimile of proof at secondary level; most students must learn proof from the basics. Critical reading 
of an essay is a related but different skill to writing an essay, and the same holds for proof. Students who 
learn to critically evaluate proofs are able to critically evaluate their own proofs as they write and, eventually, 
to write better proofs. 

With such a novel concept as proof, the potential drawbacks of peer assessment are magnified. Students 
submitting assignments are not used to writing precisely, and students providing feedback are not used to 
unpicking imprecise writing to offer constructive feedback. The likelihood of a student providing valuable 
feedback is low, especially towards the beginning of the module, even assuming they are motivated and not 
under time pressure. Nevertheless, the potential advantages of peer feedback are also amplified. Plenty of 
feedback is of great value in a module dealing with a novel means of communication. And even peer 
feedback that suggests the provider misunderstood the argument of the assessed work can be of value: it 
shows that the argument was expressed with insufficient clarity for a member of the target audience to 
follow. Critical reading of proof is itself a learning objective of the module, and other students’ work is a 
good source of proofs of varying quality. 

Handwritten mathematical notation includes many symbols that are not present on a standard computer 
keyboard, and two dimensional arrangements of symbols which are not easily reproduced using standard 
word processing software. For this reason, mathematicians use the LaTeX markup language in place of MS 
Word etc. Using LaTeX to typeset mathematics is another learning objective of the proof module. Students 
are expected to typeset all their submissions for assessment. Learning computer languages is typically 
undertaken by students in relative isolation, with only online resources to help, so students from a single 
cohort typically learn disparate techniques, many of which could benefit all students. As students are often 
reluctant to use virtual learning environment (VLE) message boards, peer feedback could be an efficient 
means for exchanging LaTeX tips. 


3. PROPOSED COLLABORATIVE FEEDBACK MODEL 

This research proposes the following model of peer feedback, designed to address some of the issues raised 
above. 

To improve the quality of feedback provided, the process of giving peer feedback is conducted 
collaboratively by a randomly assigned group of 3-4 students (the feedback group) during class time, with 
some oversight by the instructor. Students in the same feedback group will often have varying 
mis/interpretations of the work on which they are to give feedback (submitted work). It is insisted that the 
feedback group arrive at a consensus on each feedback note before recording it. In order to enforce this rule. 
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each feedback group is provided with only one pen, which must be shared, with a different student acting as 
scribe for each submitted work. With only one pen available, students are prevented from working 
individually on separate submitted works. Requiring a dialogue before feedback is recorded not only 
equalizes the quality of peer feedback generated, but also greatly raises the quality. Groups of students can 
achieve higher quality results on an assignment, and peer feedback is no different. The peer feedback 
exercise is run in class time in order to encourage students to view the exercise as a valuable learning activity 
rather than an administrative duty that may be shirked. 

As the instructor oversees multiple feedback groups simultaneously, the instructor has limited time to 
actively participate in the provision of feedback with any one feedback group. Therefore, the instructor's role 
is primarily in managing time and ensuring that the feedback groups are operating collaboratively. As expert 
advice is valuable for the feedback groups, the instructor provides model solutions and advice for the 
feedback groups to study before the submitted work is made available. A list of common errors may be 
helpful to the feedback group in providing the best feedback possible, but it was deemed to risk causing 
embarrassment to a student, therefore avoided; the proper time for a student to engage with feedback on their 
own submission is when they receive full personalised feedback, not immediately before they give feedback 
to others. 

In advance of the class, students submit their work through the VLE as a raw (computer code) LaTeX 
document. The instructor pseudononymises the submissions, compiles to pdf. and decides on the distribution 
of submitted work to feedback groups. To avoid potential student embarrassment, the instructor ensures that 
at no time will any student be providing feedback on their own submitted work. Care is also necessary to 
ensure that each feedback group will be assigned a similar amount of submitted work. After the class, the 
instructor scans the peer feedback, matches pseudonyms to the students’ submitted work, and distributes the 
feedback via the VLE. The pseudonyms are changed each time the exercise is conducted, so that students do 
not even know their own pseudonym until they receive the feedback, and at no time to do they know the 
pseudonyms of others. Therefore, the student experience of pseudonymisation is the same as if it were 
anonymisation. 

After the feedback has been distributed, students are required to assimilate the feedback they received by 
submitting a paragraph of reflection through the VLE. This is designed to enforce some engagement with the 
peer feedback, closing the learning cycle. Students are encouraged to be critical of the feedback if 
appropriate, but to bear in mind that the purpose of a proof is to communicate an argument to a peer—if this 
was unsuccessful then the responsibility is unlikely to lie wholly with the reader. 


4. IMPLEMENTATION OF THE MODEL 

The model was originally implemented in the Proof module in semester 1 of academic year 2016-2017. 
Measured by anonymous student evaluations of the module, it was found that students were generally 
enthusiastic about the learning activities, and saw the exercise as valuable. Measured by the students’ 
submissions of reflective assignments, it appeared that the students rated the feedback as valuable. We 
hypothesise that this was due to the students having seen their peers providing the feedback, and the careful 
work that went into it; knowing that their own feedback groups gave high quality feedback, students 
extrapolated to the feedback groups providing feedback to them. 

The greatest issues with the implementation were (1) the administrative burden on the instructor and (2) 
the risk that some students may not receive important feedback because all students in the feedback group 
suffer the same misconception as the student whose work was assessed. 

Administrative tasks such as sorting are well suited to automation. Prototype software was developed to 
implement this, greatly reducing problem (1). Indeed, the tasks of randomly assigning students to feedback 
groups respecting diversity, pseudonymising and compiling to pdf the submitted work, and assigning 
submitted work to feedback groups was all successfully automated. It remains to automate the process of 
matching students’ pseudonyms to submitted work and uploading the feedback to the VLE. Software is 
currently being developed to address this using barcodes. 

To address (2), the following modification was made to the model for future semesters. After the peer 
assessment is complete, the instructor asks students to share with the class the errors that they encountered 
most often in the submitted work. By noting this feedback on the board, the instructor provides generic 
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feedback that may benefit all students. An incidental benefit is reinforcing the message to students that theirs 
is not the only submitted work with errors, in advance of receiving their personalised feedback. The 
instructor also gains an, albeit limited, measure of the most common areas for improvement. It was found that 
the generic feedback exercise obviated each student receiving feedback from multiple feedback groups. 
Therefore the modified model is more efficient with class time. The generic feedback was also a particularly 
good time to address learning objectives related to LaTeX. It may be that the context of a specific 
assignment, where all students had to solve very similar typesetting problems lent itself to productive class 
discussions on typesetting. 


5. CONCLUSION 

Peer feedback as naively constructed can have significant drawbacks alongside its undeniable benefits to 
student experience and learning. The proposed model for collaborative peer feedback was successful in 
avoiding many of the most significant drawbacks of peer feedback. Increased automation will further 
improve the faculty experience of the model, without affecting the student experience. Although the model 
was implemented in a module relying on students submitting LaTeX source files, there is no significant 
impediment to implementing the fully automated model in another context. Indeed, provided all work is 
submitted through an LMS, there should be no impediment to automating the pseudonymisation. 
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ABSTRACT 

The author demonstrates a new system useful for class reflection inspired from “lesson study,” a process in which 
teachers jointly plan, observe, analyze, and refine actual classroom lessons called “research lessons”. Our new system 
offers an environment that one can use an iPad to bookmark observers’ feedbacks into simultaneously recorded videos in 
the environment. If one uses video recording and feedback check sheets in class reflection, one can easily use the system, 
and makes learning more effective and efficient because of the following two main reasons. 1. Participants instantly 
reflect on his/her performance by watching scenes supplemented with both aggregate and synchronized individual 
observer’s feedback. 2. Observers can give detailed feedback to the performer based on visual evidences and bookmarked 
feedback information, which make their learning more clear. 


KEYWORDS 

Lesson Study, Class Reflection, Class Capture, PF-NOTE 


1. DEMONSTRATION 

The author’s group has developed a system that the system uses video cameras, clickers and a computer to 
bookmark an audience’s clicker feedback (event) into simultaneously recorded video. The system named 
PF-NOTE has been mainly used for reflective learning, and we have found that this system is effective in 
various situations for both teacher and learners, such as practice teaching for early career teachers through 
class reflection (Nakajima 2008, Miura 2014). The system has two main strengths for class reflection. The 
first is instant playback function of important scenes on a session, which increases time efficiency. The 
second is sharing feedback information provided by observers, which could make performers learning more 
effective. In this paper, the author introduces a new system by extending our system that users can use iPads 
instead of clickers. By using the new system, observers can evaluate performers and share their more detailed 
feedbacks by touching on check sheets on their iPads’ screens, which are commonly used in research lessons. 
Feedbacks newly offered by the system are feedback viewpoints and detailed comments. 

The author will bring whole system and demonstrate class reflection process briefly in two minutes. 
Participants will experience the technology with using the devices the presenter bring. Several participants 
will give descriptive feedbacks to the presenter’s short presentation. Then it will be shown with the recorded 
video immediately after the presentation. 
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Figure 1. System Overview 
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Figure 2. Ipad Screen Overview 
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ABSTRACT 

In a globalized digital age, the creation of curriculum innovation, along with the way we deliver course content has a 
great impact on preparing and equipping students with the knowledge and skills needed to succeed in the workplace. The 
affordance of connectivity of Massive Open Online Courses (MOOCs) has provided new opportunities for higher 
education institutions to develop a more adaptive and strategic approach to online learning. The Industry 4.0 movement 
has reflected the needs of business educators to embrace the new trends and challenges in today’s workplaces. To better 
utilize MOOCs in business education, it is important to take note of this shift and evolution and explore how MOOCs can 
be used to support university teaching and learning. As the use of MOOCs has been widely discussed and spread 
throughout educational disciplines, business educators need to evaluate the practicality of this learning environment and 
make its applications effective, while considering its pedagogical benefits and constraints. 


KEYWORDS 

Business Education, Curriculum Design, MOOCs, Mixed Methods 


1. PURPOSE OF THE STUDY 

The goal of this study aims to provide visibility of curriculum design, implementation and evaluation as well 
as to link evidence-based educational research toward a better understanding of how students participate in 
MOOCs-supported course. To better transform business education, experimenting on combining 
competency-based curriculum with MOOCs helps facilitate integrated growth of business students through 
structured online activities in a flipped learning fashion (Al-Atabi and DeBoer, 2014; Li et al, 2015). Thus, 
we implemented a blended use of MOOCs in business classrooms with the hope that the blended usage 
bridges the gap between existing curriculum design and space in making a difference regarding learners’ 
acquisition of relevant knowledge, competence and skills (Huang and Lin, 2017; Liu et al, 2014; 
Liyanagunawardena et al, 2013). This study investigated the following two questions: (1) How can MOOCs 
support university-level business curriculum design and accommodate different learning needs?, and (2) 
What are the learner perceptions and attitudes towards MOOCs use within the context of a university-level 
business course? 


2. METHOD AND CONCLUSION 

Mixed methods were employed to assess the data from both qualitative and quantitative perspectives 
(Johnson and Onwuegbuzie, 2004). The survey questionnaire and open-ended questions were adopted and 
modified from the conceptual framework. Unified Theory of Acceptance and Use of Technology, taking into 
account the learning environment in which MOOCs are adopted. Data collected from 120 undergraduate 
business students in a Taiwanese university coupled with their written reflections about the course experience 
were analyzed. The findings suggested that the practicality of blending MOOCs has to do with their course 
content design and also with their alignment to the existing curriculum objectives. Learners’ intention to use 
MOOCs is influenced in particular by social influence and facilitating conditions. The study concludes with a 
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consideration of both the benefits and the challenges pertaining to this new approach of applying MOOCs for 
quality business education. In order to advance this mission, future research studies are expected to 
incorporate relevant learning community-based and psychological factors in demonstrating course quality 
and operational effectiveness of MOOCs. 
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ABSTRACT 

Gamification is a strategy of using game mechanics and experience design to digitally engage people to achieve intended 
goals in non-game contexts. There has been increasing discussion among educators and researchers about harnessing the 
idea of gamification to enhance current learning and teaching practices in school education. This paper presents our 
initiative to gamify outdoor social inquiry learning with context-aware technology, as well as briefly reporting its 
pedagogic effectiveness measured in a quasi-experimental study. 


KEYWORDS 
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1. BACKGROUND 

Social inquiry learning (SIL), which has been considerably adopted in social humanities education, 
emphasises on students’ inquiry into humans and their relationships with the “societal globe” from multiple 
perspectives, values and interests (Chadwick, 2008; Hill, 1994; Jansen, 2011). Thus, in the course of SIL, 
situating and engaging students to interact with real-life, real-world contexts is important (Stripling, 2003, 
2008). Curiosity is the best driving force for learning; keeping learners curious via engaging them in 
game-based activities is a desirable approach to education (Papert, 1993; Piaget, 1970). Gamification refers 
to the use of game elements in non-gaming contexts (Deterding, Dixon, Khaled & Nacke, 2011). Lee and 
Hammer (2013) have further defined gamification as the incorporation of game mechanics into non-gaming 
software applications to promote user experience and engagement. 


2. OUR INITIATIVE 

Guided by Stripling’s (2003, 2008) social inquiry model and Lee et aids (2013) gamification model, we have 
developed a GPS-supported gamified learning mobile application (App) (see Figure 1) for engaging and 
supporting students to pursue context-aware SIL in outdoor environments. Based the received GPS signals, 
gamified context-aware scaffolds of the App will pop up in accordance with students’ physical positions in 
the real world. The scaffolds will engage and support students in accomplishing the outdoor inquiry tasks at 
the fieldtrip site. At each inquiry spot, the Connection scaffold (see Figure 2), which is in the form of voice 
navigation, will provide students with background information about the societal issue to be inquired. The 
Investigation scaffold, which is in the form of data-collection exercise, will guide students to gather new 
information to answer the questions related to the issue. The Construction scaffold will assist students in 
generalising the information on hand via mind-mapping and making an interim conclusion about the issue via 
audio-recording. The Reflection scaffold will support students, via video-blogging, in reflecting on their 
weaknesses and setting new goals for inquiring the next spot. After completing the tasks at a spot, students 
will be awarded a “star.” The leader board (see Figure 3) in the App will dynamically indicate how many 
stars each student has obtained and how much time he/ she has spent on obtaining the stars in a real-time 
manner. 
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Figure 1. Interface of the App 



Figure 2. Connection Scaffold 



Figure 3. Leader Board 


3. RESEARCH FINDINGS 

A quasi-experimental study was carried out to investigate the pedagogic effectiveness of the proposed 
“gamified” S1L (viz. GSIL) approach in comparison with conventional S1L approach, in terms of students’ 
knowledge acquisition. It involved a total of 373 Grade-10 students from top, middle, and bottom 
academic-banding schools; 128 were high-achieving students, 127 were moderate-achieving students, and 
118 were low-achieving students. Results indicated that, compared to the conventional approach, GSIL had 
different degrees of significant positive effects on the high, moderate, and low academic-achieving students 
(Cohen’s d effect sizes: 0.30, 0.62, and 1.04). 
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